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Section  1 


INTRODUCTION 

BACKGROUND 

Aircraft  accidents  are  infrequent  occurrences  but  when  they  do 
occur  they  result  in  loss  of  life.  Pilots  are  usually  implicated  as  a 
causal  factor  in  half  of  these  accidents  (Boeing,  1985) .  Of  the 
percentage  of  accidents  that  do  occur,  over  55%  happen  during  the 
approach  and  landing  phases  of  flight  (Boeing, 1985) .  The  present 
research  shall  examine  pilot  performance  during  an  approach-to-landing 
task  with  two  different  instrument  landing  system  displays.  To  aide  the 
reader  in  understanding  the  problem  of  conducting  an  approach-to-landing 
without  visual  reference,  a  brief  historical  overview  of  aircraft 
instrument  design  and  integration  is  provided.  This  is  followed  by  a 
brief  overview  of  the  Instrument  Landing  System,  aircraft  flight 
director,  and  models  of  pilot  control  strategy.  Finally,  background  on 
the  B-1B  flight  station  and  Instrument  Landing  System  displays,  used  as 
the  research  vehicle  in  this  study,  is  provided  and  the  specific 
research  hypothesis  identified. 

Aircraft  navigation  by  reference  to  radio  wave  steering 
information  has  existed  since  the  1930's.  The  equipment  of  that  era 
consisted  of  an  aircraft  receiver  that  conveyed  steering  information  to 
the  pilots  by  use  of  Morse  code  provided  in  their  headsets  (Wiener  & 
Nagel, 1988).  During  the  1940 's  and  1950 's,  as  technology  progressed, 
more  precise  instrumentation  replaced  the  Morse  code  based  Adcock  range 
(Cooper,  1991) .  New  navigation  systems  provided  both  auditory  and 
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visual  guidance  information  to  the  pilot  for  both  enroute  and  terminal 
portions  of  flight.  The  primary  navigation  instrument,  called  a  Course 
Deviation  Indicator,  informed  the  pilot  if  he  was  on  centerline  or 
left/right  of  the  navigation  beam. 

Until  the  Fitts,  Jones,  and  Milton  (1942)  studies  of  instrument 
scanning,  aircraft  manufacturers  placed  instruments  where  space  was 
available.  The  results  of  Fitts  f£.  al .  changed  the  way  instruments  were 
arranged  in  civilian  and  military  aircraft.  The  ''t-andard  layout  that 
was  found  to  be  effective  in  reducing  pilot  workload  is  called  the 
"Sacred  Six"  or  "T"  instrument  layout  (Ercoline,  1985) . 

One  of  the  findings  by  Fitts  £l .  was  that  certain  instruments 
are  viewed  more  frequently  than  others.  Fitts  sL  3l .  proposed  that  if 
they  could  combine  frequently  used  instruments  into  one,  they  would 
improve  pilot  performance  and  also  save  critical  instrument  space.  One 
of  the  first  instrument  integrations  was  the  combination  of  the  Course 
Deviation  Indicator  also  known  as  a  Cross  Pointer  and  the  Directional 
Gyro  resulting  in  the  creation  of  the  Horizontal  Situation  Indicator. 
Illustrated  in  Figure  1  is  the  original  eight  instruments  used  in  the 
Fitts,  fat  al .  study.  The  basic  six  instruments  arid  their  placement  are 
the  two  rows  of  three  instruments.  They  are:  Airspeed,  Directional 
Gyro,  Gyro  Horizon,  Altimeter,  Turn  and  Bank  and  Vertical  Speed 
indicators . 
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Aircraft  instruments  that  are  used  for  instrument  flight  today 
are:  1)  the  Airspeed  Indicator,  2)  the  Directional  Gyro  or  Horizontal 
Situation  Indicator,  o)  Attitude  Indicator  also  known  as  the  Gyro 
Horizon,  4)  the  Altimeter,  f,  the  Turn  and  Bank  Indicator,  and  6)  the 
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Vertical  Velocity  Indicator. 

Airspeed  information  is  combined  with  timing  information  on  a 
landing  approach  and  is  used  as  a  cross  check  of  position  during  the 
approach.  A  specific  airspeed  is  also  used  as  a  target  setting  during 
the  approach  depending  on  the  gross  weight  of  the  aircraft .  The 
Horizontal  Situation  Indicator  or  Directional  Gyro/Course  Deviation 
Indicator  combination  provides  the  pilot  with  the  present  heading, 
direct  of  turn,  and  steering  information.  Gyro  horizon  or  aircraft 
attitude  information  informs  the  pilot  of  the  aircraft  attitude  in  both 
pitch  and  roll  axes.  Altimeter  information  provides  the  corrected 
absolute  reference  for  the  starting  and  decision  points  during  an 
approach-to-landing.  The  Turn  and  Bank  Indicator  provides  a 
confirmation  of  the  direction  the  aircraft  is  turning  and  if  the 
aircraft  is  flying  in  a  coordinated  manner.  Vertical  velocity 
information  informs  the  pilot  of  his  rate  of  climb  or  descent. 

It  was  noted  by  Fitts  <=£.  a]  .  that  during  instrument  approaches, 
pilots  spent  considerable  time  scanning  both  the  Course  Deviation 
Indicator  and  the  Dir?ctional  Gyro.  Both  of  these  instruments  were 
combined  into  one  instrument,  called  the  Horizontal  Situation  Indicator 
(Williams  &  Roscoe, 1949) ,  which  resulted  in  pilots  spending  less  time 
scanning  and  hence  seemingly  lowering  pilot  workload.  The  Horizontal 
Situation  Indicator  and  the  Attitude  Direction  Indicator  are  juxtaposed 
so  as  to  keep  the  critical  information  in  foveal  vision.  According  to 
Kelley  (1968)  and  Roscoe  (1968)  this  instrument  placement  improved  the 
ability  of  the  pilot  to  notice  and  respond  to  velocity  changes  on  a 
display.  Control  of  a  multi-axis  vehicle,  such  as  an  airplane,  using  an 
Attitude  Direction  Indicator  is  more  natural  (aircraft  direction  of 


movement  is  the  same  direction  as  the  control  input)  using  the 
integrated  displays.  As  aircraft  continued  to  become  larger  and 
heavier,  pilot  workload  increased  due  to  larger  aircraft  inertia  forces 
and  corresponding  slower  control  system  response  characteristics.  Snail 
deviations  if  not  corrected  immediately  result  in  large  excursions  from 
the  desired  flight  path.  Recovery  back  to  the  correct  flight  path  would 
require  a  large  control  input  resulting  in  a  degraded  overall  system 
performance  with  the  possibility  of  causing  pilot-induced-oscillations. 
Complicating  the  problem  of  slow  system  response  was  that  the  pilot  was 
responsible  for  integrating  all  instrument  information  and  then  deciding 
on  the  correct  course  of  action  to  maintain  flight  path.  The  standard 
instrument  layout  does  not  directly  inform  the  pilot  of  a  deviation  from 
the  flight  path.  Once  the  deviation  has  been  detected,  the  pilot  acts 
as  a  rate  controller  and  must  estimate  the  correction  required.  However 
for  large  aircraft,  a  time  delay  is  resident  in  the  system  response 
before  it  reacts  to  a  control  input. 

The  time  it  takes  a  pilot  to  integrate  information  across 
displays,  coupled  with  a  control  response  lag,  is  the  principal  reason 
that  the  aircraft  flight  director  was  invented.  The  flight  director 
consists  of  a  computer  that  receives  signals  from  aircraft  navigation 
and  performance  instruments  and  displays  an  optimized  steering  path  for 
the  human  pilot  or  autopilot  to  follow.  The  flight  director  was 
designed  to  minimize  flight  path  deviations  and  reduce  pilot  workload. 
The  flight  director  computer  can  detect  small  deviations  almost 
immediately  and  provide  a  steering  correction  for  the  pilot  to  follow 
thus  reducing  his  integration  burden  and  reducing  the  likelihood  of 
pilot-induced-oscillation.  The  piloting  task  is  then  reduced  to  a  two- 
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dimensional  tracking  task.  The  purpose  of  this  study  centers  around  the 
information  provided  by  the  flight  director  with  and  without  raw 
steering  information.  Of  interest  is  how  a  pilot  might  use  this 
information  during  an  approach-to-landing  task. 

The  integration  of  the  flight  director  occurred  shortly  after  the 
analog  computer  was  miniaturized  to  fit  in  fighter  sized  aircraft 
(Birmingham  &  Taylor,  1954) .  The  flight  director  design  was  a  spin-off 
of  research  being  performed  by  the  U.  S.  Air  Force  on  fighter  aircraft 
and  on  submarines  by  Birmingham  and  Taylor  (1954)  for  the  U.  S.  Navy. 
Typically,  the  steering  information  provided  by  the  flight  director  is 
presented  on  the  Attitude  Direction  Indicator  as  a  steering  cross 
representing  steering  commands  for  pitch  and  roll  axes.  The  flight 
director  receives  the  navigational  information  from  the  Instrument 
Landing  System  located  adjacent  to  the  runway.  The  steering  information 
provided  by  the  Instrument  Landing  System  consists  of  two  steering  beams 
representing  localizer  and  glideslope.  Localizer  and  glideslope 
steering  information  is  presented  in  an  unfiltered,  raw  form  on  the 
Horizontal  Situation  Indicator  and  in  a  filtered  form  on  the  steering 
cross  of  the  Attitude  Direction  Indicator.  Figure  2  illustrates  the 
typical  representation  of  the  Attitude  Direction  Indicator  with  steering 
cross  and  the  Horizontal  Situation  Indicator  with  raw  glideslope  and 
localizer  data. 
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.FLIGHT  DIRECTOR  SYSTEM 


Figure  2. Typical  Flight  Director,  the  top  figure  is  an  Attitude 
Direction  Indicator  with  steering  cross,  the  bottom  figure  is  the 
Horizontal  Situation  Indicator 

For  a  more  complete  understanding  of  the  flight  director  and  the 
associated  use  of  raw  data,  examination  of  the  key  elements  of  an 
Instrument  Landing  System  are  in  order.  A  typical  Instrument  Landing 
System  must  include  instrumentation  that  provides  precise  steering  to 
the  runway  environment  and  precise  steering  down  the  Glideslope  beam 
that  provides  the  vertical  steering  cues  to  the  pilot.  As  can  be  seen 
in  Figure  3,  the  Instrument  Landing  System  provides  a  narrow  steering 
beam  for  lateral  runway  alignment  of  approximately  five  degrees.  This 
beam  represents  the  localizer  and  is  aligned  within  a  few  degrees  of  the 
runway.  Vertical  guidance  is  provided  by  a  very  narrow  glideslope  beam 
of  approximately  one  and  four  tenths  of  a  degree.  The  aircraft  receives 
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the  steering  information  from  the  Instrument  Landing  System  and 
depending  upon  how  the  aircraft  is  equipped,  can  represent  the  steering 
information  on  either  a  Course  Deviation  Indicator,  a  Horizontal 
Situation  Indicator,  or  a  flight  director  equipped  with  an  Attitude 
Direction  Indicator  and  Horizontal  Situation  Indicator.  The  pilot's 
task  is  the  same,  independent  of  the  method  of  Instrument  Landing  System 
presentation.  The  piloting  task  is  to  keep  the  steering  needles  in  the 
center  of  the  display  which  represents  for  all  displays  the  center  of 
the  steering  beams. 


I  OCAUZER 


GUOESIOPE 


Figure  3.  ILS  BEAM  WIDTHS,  top  figure  is 
Localizer,  bottom  figure  is  Glideslope. 
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THE  INSTRUMENT  LANDING  SYSTEM 


An  Instrument  Landing  System  must  provide  distance  or  positional 
information  from  the  runway  threshold.  These  distances  or  positions 
must  be  identified  and  correspond  with  key  elements  of  this  system  such 
as  Decision  Height  or  Glideslope  Intercept  Point.  Decision  Height  is 
usually  defined  as  an  altitude  of  200  feet  above  Mean  Sea  Level  which 
corresponds  to  approximately  one-half  mile  from  the  end  of  the  runway. 
Decision  Height  serves  as  the  point  where  the  pilot  must  decide  to 
either  land  the  aircraft  or  conduct  a  missed  approach  if  the  runway  or 
airport  environment  is  not  visible.  The  Glideslope  Intercept  Point  is 
located  at  the  beginning  of  the  approach  and  is  the  point  that  the 
aircraft  begins  descending  based  on  vertical  steering  cues  provided  by 
the  glideslope  beam.  Both  Decision  Height  and  Glideslope  Intercept 
Point  can  be  represented  by  either  a  fixed  location  marker  or  by 
Distance  Measuring  Equipment,  measured  in  nautical  miles.  Fixed 
markers,  known  as  marker  beacons,  are  represented  in  a  flight  station  by 
an  annunciator  of  flashing  lights  and  an  aurally  provided  Morse  code 
identifier  in  a  pilot  headset  or  aircraft  speaker.  Distance  Measuring 
Equipment,  shown  in  Figure  4,  is  displayed  as  a  digital  readout  on  a 
Horizontal  Situation  Display  or  may  be  a  separate  instrument.  Distance 
Measuring  Equipment  provides  distance  in  nautical  miles  from  a  known 
reference  point. 
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Figure  4.  DISTANCE  MEASURING  EQUIPMENT,  Horizontal  Situation 
Indicator  with  a  distance  of  2.0  nautical  miles  shown  in  the  upper  left 
comer 


The  basic  instrumentation  convention  for  both  military  and 
civilian  aircraft  today  has  changed  little  since  the  1950’s.  The 
implementation  of  the  Cathode  Ray  Tube  into  the  flight  station  occurred 
in  1968  when  Sobocinski  (1968)  successfully  integrated  a  working 
prototype  of  a  Horizontal  Situation  Indicator  into  a  flight  test 
aircraft.  The  availability  of  the  Cathode  Ray  Tube  has  released  the 
aircraft  engineer  and  designer  from  employing  designs  that  were  required 
due  to  limitations  of  an  electromechanical  instrument.  Primary  flight 
information  such  as  that  provided  by  an  Attitude  Direction  Indicator  or 
steering  information  provided  by  the  Horizontal  Situation  Indicator 
could  now  be  displayed  in  any  manner  that  the  designer  wishes.  This 
unbounded  design  option  offered  challenges  to  designer,  researcher  and 
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pilot  alike.  Questions  concerning  the  method  of  presenting  data  to 
pilots  now  had  greater  significance.  Previous  research  questions  that 
attempted  to  determine  what  should  move  on  a  display,  where  it  should  be 
located  and  when  should  it  move  (Berube,  1981)  will  require  reevaluation 
as  new  Cathode  Ray  Tube  displays  replace  their  electromechanical 
predecessors  in  all  class  and  categories  of  aircraft.  Cathode  Ray  Tube 
display  research  on  aircraft  predictor  displays  has  been  successfully 
transitioned  and  implemented  into  Boeing  757  and  767  cockpits  (Jensen, 
1981/  Grunwaid,  1981/  and  Palmer,  Jago,  &  DuBord,  1981) .  Other 
researchers  (Aretz,  1988,  1989/  Harwood,  1989/  Wickens,  Aretz  &  Harwood, 
1989)  have  examined  Head  Down  Displays  in  the  context  of  situational 
awareness  and  pilot's  internal  frame  of  reference.  Implementation  of 
newer  display  designs  into  civilian  and  military  aircraft  will  be 
difficult.  Airline  owners  are  very  conservative  and  are  not  willing  to 
irrplement  newer  technology  unless  that  technology  is  better  and  of  lower 
cost  than  that  presently  in  service.  Pilots  also  are  not  likely  to 
receive  this  new  technology  in  a  positive  light  until  issues  such  as 
transfer  of  training  are  addressed. 

B-1B  FLIGHT  STATION  DISPLAY  FORMATS 

The  B-1B  is  capable  of  executing  Instrument  Landing  System 
approaches  in  almost  any  weather.  The  B-1B  Instrument  Landing  System 
displays  used  by  the  pilots  to  fly  these  approaches  have  required 
redesign  due  to  poor  representation  of  critical  data  presented  on  the 
Vertical  Situation  Display.  The  first  B-1B  Instrument  Landing  System 
display  was  a  transformed  version  of  the  one  used  on  the  Space  Shuttle. 
Rockwell  International,  prime  contractor  of  both  the  Space  Shuttle  and 
the  B-1B,  employed  the  same  Instrument  Landing  System  display  used  by 
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the  Space  Shuttle  into  the  B-1B  aircraft.  Aircraft  differences  and 
operational  environment  dissimilarities  resulted  in  poor  user  acceptance 
of  the  Space  Shuttle  Instrument  Landing  System  box.  The  Space  Shuttle 
Instrument  Landing  System  display  required  the  pilot  to  fly  the  flight 
director  steering  cross  into  the  center  of  a  box  representing  the  center 
of  the  Instrument  Landing  System  (see  Figure  5) .  The  redesigned 
display  uses  fixed  dots  to  represent  glideslope  and  localizer  position 
and  is  similar  to  the  Instrument  Landing  System  that  is  in  current  Air 
Force  aircraft. 


Figure  5.  B-1B  INSTRUMENT  LANDING  SYSTEM  DISPLAY 
The  B-1B  flight  station  was  designed  as  a  conventional  cockpit 
except  for  a  Cathode  Pay  Tube  display  located  directly  in  front  of  each 
pilot.  The  Cathode  Ray  Tube  displays  primary  flight  information  and  is 
called  a  Vertical  Situation  Display.  The  B-1B  aircraft,  as  presently 
configured,  provides  the  Instrument  Landing  System  navigational 
information  to  each  pilot  through  the  Vertical  Situation  Display 
(Reynolds,  Purvis,  &  Marshak,  1990) .  Primary  flight  navigation 
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information  such  as:  attitude,  airspeed,  altitude,  heading  and  steering 
information  is  provided  on  this  display.  The  B-13  Vertical  Situation 
Display  can  be  considered  an  integral,  object  display.  An  integral 
object  display  according  to  Wickens  (1984)  provides  the  most  compatible 
means  of  providing  the  supervisor  (in  our  application,  pilot)  with  an 
overall  representation.  The  small  airplane  symbol  on  the  B-1B  Vertical 
Situation  Display  remains  fixed  in  the  center  of  the  display  relative  to 
the  world  around  it.  Sky/ground  shading  and  movement  of  pitch  and  roil 
axes  are  represented  to  the  pilot  from  an  inside  the  aircraft 
perspective  looking  outside.  The  representation  of  the  glideslope  and 
localizer  beams  and  locations  during  an  instrument  approach  to  landing 
are  represented  by  the  dots  and  cursor  combination  on  the  Vertical 
Situation  Display. 

One  of  the  many  design  requirements  for  this  aircraft  was  that  it 
be  capable  of  precision  approaches  using  earth  referenced  navigational 
sources  such  as  an  Instrument  Landing  System  provides.  The  B-1B 
presents  attitude  information  on  the  Vertical  Situation  Display  with  a 
fixed  aircraft  symbol  and  a  moving  horizon,  thus  the  display  is 
aircraft-referenced.  The  B-1B  displays  airspeed  information  to  the 
pilot  on  the  Vertical  Situation  Display  in  either  indicated,  true,  or 
ground-referenced  (see  Appendix  A) .  Indicated  airspeed  is  used  for  an 
approach-to-landing.  Altitude  information  is  provided  in  either  an 
absolute  term,  Above  Ground  Level  or  referenced  to  a  sea  level  datum 
called  Mean  Sea  Level.  Heading  information  is  the  direction  the 
aircraft  is  traveling  with  either  an  earth  reference  to  true  or  magnetic 
North. 

The  B-1B  flight  station  Vertical  Situation  Display  as  seen  in 
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Figure  6  is  unique.  It  is  the  only  aircraft  known  to  integrate  both  an 
Attitude  Direction  Indicator  and  Horizontal  Situation  Indicator 
information  on  the  same  Head  Down  Display. 


VSOILS  VMIhXTj 


Figure  6.  VERTICAL  SITUATION  DISPLAY 
FLIGHT  DIRECTOR  DESIGN 

Until  the  D-1B  bomber  became  operational,  all  flight  directors 
contained  both  flight  director  steering  information  and  raw  glideslope 
and  localizer  information.  Electromechanical  system  designs  dictated 
that  if  the  primary  steering  information  wos  not  available  due  to  either 
a  component  failure  or  indicator  malfunction,  flight  crews  could  fly  the 
approach-to-landing,  albeit  less  precisely  with  the  raw  data  display. 

The  B-1B  flight  station  Vertical  Situation  Display  Instrument 
Landing  System  was  originally  designed  without  ^ny  raw  glideslope  or 
localizer  data/  only  flight  director  steering  information  was  provided. 
Recently  the  B-1B  flight  station  Vertical  Situation  Display  formats, 
used  for  the  display  of  both  navigation  and  status  information  were 
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redesigned  to  incorporate  raw  glideslope  and  localizer  information 
(Purvis,  Green,  St.  John,  Reynolds,  &  Lovering,  1988).  Aircraft  that 
contain  multiple  Cathode  Ray  Tube  systems  are  typically  called 
Electronic  Flight  Instrument  System  aircraft.  However,  unlike  the  B-1B, 
previous  commercial  and  military  Electronic  Flight  Instrument  System 
displays  have  not  changed  in  orientation  or  color  from  that  of  their 
older  analog  counterparts,  an  electronic  Attitude  Direction  Indicator 
looks  like  an  electromechanical  Attitude  Direction  Indicator.  All 
currently  produced  Electronic  Flight  Instrument  System  displays  provide 
the  same  Horizontal  Situation  Indicator  layout  with  raw  glideslope  and 
localizer  information  although  that  information  is  no  longer  required  by 
the  FAA  to  be  on  the  display. 

B-1B  pilots  are  presented  with  steering  commands  that  must  first 
be  integrated  by  the  flight  director  computer  before  they  are  presented 
on  the  Vertical  Situation  Display.  As  can  be  seen  in  Figure  7,  the 
flight  director  control  panel  must  be  in  one  of  the  active  flight 
director  modes  before  any  steering  commands  are  provided  to  the  Vertical 
Situation  Display.  When  the  flight  director  panel  is  in  the  off 
position  (see  Figure  7) ,  the  steering  bars  are  removed  (stowed)  and  no 
steering  information  is  presented. 
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Figure  7.  VERTICAL  SITUATION  DISPLAY  WITH  FLIGHT  DIRECTOR 
The  B-1B  flight  director  provides  steering  information  to  the 
pilot  in  the  shape  of  a  steering  cross.  The  flight  director  steering 
cross  only  provides  steering  information  during  the  Instrument  Landing 
System  approach.  The  raw  glideslope  and  localizer  data  presented  on  the 
Vertical  Situation  Display  in  dot  form,  reinforce  or  confirm  the  flight 
director  steering  commands.  Typically  a  flight  instrumentation  system 
provides  this  raw  data  in  the  event  that  a  mechanical  failure  disables 
the  flight  director.  However  digital  display  systems  typically  require 
all  data  to  be  input  into  the  flight  director  system  before  display 
presentation.  This  makes  the  raw  data  display  redundant  if  used  as  a 
back-up  to  the  flight  director.  If  a  flight  director  component  or 
Vertical  Situation  Display  fails,  no  partial  degraded  capability  exists 
and  whatever  backup  analog,  electromechanical  instrument  is  available 
becomes  the  only  means  of  presenting  navigation  information. 
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The  Vertical  Situation  Display  presents  raw  glideslope  and 


localizer  information  as  a  series  of  fixed  dots  that  represent  the 
beamwidth  of  the  guidance  signals.  Figure  8  depicts  this  display.  The 
glideslope  dots  (arrow  A)  are  located  on  the  left  side  of  the  screen  and 
the  localizer  dots  (arrow  A)  are  located  on  the  bottom  of  the  screen. 

The  moving  triangular  symbol  represents  the  center  of  that  beam  and  the 
dots  are  fixed. 


Figure  8.  VERTICAL  SITUATION  DISPLAY  WITH  RAW 
GLIDESLOPE  AND  LOCALIZER  DOTS 


Illustrated  in  Figure  9,  arrow  A  points  to  the  location  of  the 
Horizontal  Situation  Indicator.  Notice  that  it  is  located  well  below 
the  Vertical  Situation  Display,  arrow  C.  The  B-1B  Horizontal  Situation 
Indicator  provides  only  lateral  steering  information,  no  glideslope 
information  is  provided.  The  Horizontal  Situation  Indicator  serves  as  a 
partial  electromechanical  back-up  to  the  Vertical  Situation  Display. 

The  Vertical  Situation  Display  is  considered  the  primary  navigational 
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instrument  for  instrument  flight.  The  Horizontal  Situation  Indicator 
backup  would  be  suitable  for  only  non-precision  approaches  due  to  the 
lack  of  vertical  guidance  information. 


Figure  9.  B-1B  FLIGHT  STATION 
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The  B-1B  is  provided  with  both  Distance  Measuring  Equipment  and 
marker  beacons  as  sources  of  information.  The  design  philosophy  was  to 
provide  the  pilot  with  a  known  location  from  which  he  could  maintain 
situational  awareness  during  an  approach-to-landing.  If  for  example, 
the  Instrument  Landing  System  glideslope  beam  fails  during  the  approach, 
the  pilot  must  revert  to  a  non-precision  approach  and  rely  upon  the 
marker  beacons  and  Distance  Measuring  Equipment  to  provide  both  timing 
and  positional  information.  A  non-precision  approach  requires  a 
different  technique  than  that  used  for  a  precision  Instrument  Landing 
System.  However  both  precision  and  non-precision  approaches  will  demand 
the  same  level  of  concentration  from  the  pilot.  The  pilot  must  be  able 
to  predict  what  the  state  of  the  aircraft  is  and  where  the  runway 
environment  will  be.  The  paper  tool  that  the  pilot  uses  to  visualize 
the  approach-to-landing  is  called  an  approach  plate.  The  approach  plate 
is  also  used  as  an  aid  for  prediction  and  is  provided  in  a  plan  and 
profile  view  of  topography  surrounding  the  airfield.  A  typical 
Instrument  Landing  System  approach  plate  is  seen  in  Figure  10.  Both 
plan  and  profile  views  of  the  Instrument  Landing  System  are  shown  to 
scale  unless  otherwise  noted. 
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Figure  10.  INSTRUMENT  APPROACH  PLATE 


Every  Instrument  Landing  System  approach  is  both  similar  and 
unique.  Each  Instrument  Landing  System  has  a  localizer  and  glideslope 
beam  that  become  narrower  the  closer  the  aircraft  is  to  the  landing 
threshold.  Each  Instrument  Landing  System  consists  of  a  gradual  descent 
of  approximately  3  degrees  to  a  minimum  altitude,  the  Decision  Height. 
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However  variables  such  as  wind  velocity  and  direction,  Glide  Slope 
Intercept  Point,  glideslope  angle  of  descent,  direction/distance  of 
localizer  intercept,  surrounding  topography,  and  levels  of  other  air 
traffic  are  always  unique.  The  B-1B  steering  cross  represents  command 
guidance  inputs  provided  by  the  flight  director  computer.  The 
Instrument  Landing  System  display  provided  on  the  Vertical  Situation 
Display  is  of  the  pursuit  type,  with  the  input  or  steering  cross 
separated  from  the  output  or  aircraft  present  position. 

The  Vertical  Situation  Display  has  a  fixed  aircraft  symbol  and  a 
moving  flight  director  cross.  The  pilot  steers  the  aircraft  symbol  in 
the  direction  of  the  steering  cross  to  reduce  the  error  between  input 
and  output.  The  principal  advantage  of  separating  input  and  output 
signals  (Garg,  1988)  is  that  the  pilot  is  better  able  to  monitor  the 
error  differential  and  anticipate  his  or  her  input.  Another  purported 
strength  of  this  display  is  that  it  permits  more  accurate  tracking 
except  when  frequencies  are  very  low  Kelley  (1968) . 

STATUS  AND  COMMAND  INFORMATION 

The  design  of  aircraft  navigation  displays  basically  falls  into 
two  categories,  those  that  are  designed  as  status  displays  and  those 
that  are  designed  as  command  or  "quickened"  displays. 

A  status  display  is  one  that  informs  the  human  operator  hay  the 
controlled  element  is  responding.  Status  displays  do  not  provide  the 
pilot  information  on  hoy  to  control  the  system.  The  pilot  must  first 
scan  and  interpret  this  information  then  act  to  control  or  modify  the 
system  state.  For  most  aircraft,  status  information  is  represented  by 
round  dial  instruments  that  indicate  airspeed,  altitude,  attitude, 
vertical  velocity,  heading,  bank  angle,  glideslope  and  localizer.  The 
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B-1B  Vertical  Situation  Display  however,  displays  airspeed,  attitude, 
heading,  glideslope  and  localizer  information  in  one  central  location. 
The  B-1B  Vertical  Situation  Display  dot  representation  of  glideslope  and 
localizer  presents  the  pilot  with  the  actual  electronic  beamwidths  and 
the  current  aircraft  location  relative  to  those  beamwidths.  Status 
information  provides  an  indication  of  how  the  overall  system  is 
performing.  The  glideslope  and  localizer  dots  provides  this  overall 
system  check  during  the  approach-to- landing  task. 

A  command  display  differs  from  a  status  display  in  that  it  informs 
the  operator  hsy  the  aircraft  is  to  be  controlled  and  provides  no 
information  as  to  the  current  aircraft  state.  Aircraft  flight  directors 
are  the  most  common  form  of  command  displays.  The  pilot  flying  an 
approach  to  landing  task  with  the  aid  of  a  flighc  director  has  only  to 
steer  the  aircraft  in  the  direction  that  the  flight  director  indicates 
for  a  successful  outcome.  Only  tracking  skill  is  required,  no  pilot 
interpretation  is  necessary.  Command  displays  require  operators  to 
receive  very  little  training  before  being  able  to  use  the  display 
(Kelley,  1968)  .  For  a  more  in  depth  review  of  coirmand  and  status 
display  types  see  Kelley  (1968)  or  Poulton  (1974) . 

Our  interest  is  in  determining  which  display  provides  better 
performance  on  an  approach-to- landing  task,  a  command  display  or  a 
command  plus  status  display.  According  to  Kelley  (1968)  and  Garg 
(1988) ,  the  advantage  of  providing  the  actual  error  (status  information) 
with  command  steering  information  is  that  better  true  performance  will 
result,  with  less  r^ot  mean  square  error.  The  results  of  flight 
simulation  work  performed  by  Garg  (1988)  indicate  that  augmenting 
command  information  with  status  information  improved  the  performance 
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(reduced  root  mean  square  error)  of  pilots  for  levels  of  display 
quickening  between  0.032  and  0.373  cm/deg-sec. 

A  pilot  flying  an  Instrument  Landing  System  approach  is 
concentrating  on  correcting  deviations  from  glideslope  and  localizer 
centerline  which,  according  to  control  behavior  models  proposed  by 
Kelley  (1968) ,  Poulton  (1974)  and  Rasmussen  (1981) ,  are  inner-loop 
activities.  Inner-loop  activities  are  those  activities  that  are 
necessary  for  maintenance  of  a  control  system.  Kelley  (1968)  has 
described  this  as  part  of  a  hierarchical  control  loop  process  with  the 
foundation  being  the  inner-loop  control  process.  In  aviation,  pitch 
controls  vertical  attitude,  vertical  attitude  controls  vertical 
displacement  from  flight  path.  Pitch  and  roll  control  are  considered 
inner-loop  processes  or  as  stated  by  Kelley  (1968) ,  zero-order  control 
loops.  A  zero-order  control  loop  is  the  lowest  control  order  in 
Kelley’s  (1968)  hierarchy.  Three  control  orders  are  possible,  zero- 
order,  first-order,  and  second-order.  A  first-order  control  process 
such  as  vertical  attitude  determines  the  second-order  control  process  of 
vertical  displacement  from  flight  path.  An  inner-loop  control  process 
can  be  a  zero,  first,  or  second  order  system.  Usually  zero  or  first 
order  systems  are  selected  as  inner-loop  control  processes.  Poulton 
(1974)  has  demonstrated  that  the  human  is  best  suited  to  tracking 
position  (zero  order)  or  velocity  (first  order)  systems  and  poorly 
suited  to  tracking  acceleration  (second  order)  systems. 

Outer-loop  activities  are  considered  by  Poulton  (1974)  and  Kelley 
(1968)  to  be  the  output  of  the  inner-loop  activities.  Outer-loop 
activities  can  be  first  or  second  order  control  processes.  A  pilot 
flying  an  Instrument  Landing  System  approach  is  typically  provided  with 
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an  assigned  heading  that  is  maintained  until  intercepting  the  localizer 
course.  Tne  inner-loop  control  process  (zero  order)  of  banking  the 
aircraft  will  change  the  outer-loop  control  process  (first  order)  of 
aircraft  heading.  However  once  the  aircraft  is  established  on  the 
localizer  beam,  aircraft  flight  path  becomes  the  outer-loop  activity 
which  is  dependent  upon  aircraft  heading.  Control  of  glideslope  and 
localizer  are  outer-loop  activities.  Kelley  (1968)  considers  this 
linear  ordering  of  control  loop  activities  to  be  consistent  with  the 
pilots  internal  model  for  the  variable  under  control.  Instrument 
student  pilots  are  provided  with  this  hierarchical  model  for  roll  and 
pitch.  As  their  training  progresses  they  understand  the  cause  and 
effect  of  their  control  inputs.  Once  these  control  loop  activities  are 
well  practiced,  the  lower  order  control  loops  become  automatic 
(Rasmussen,  1981) .  Additional  attentional  capacity  is  then  made 
available  to  the  pilot  for  monitoring  and  control  of  higher  order 
(outer-loop  )  control  activities.  Rasmussen's  (1981)  model  of  operator 
performance  agrees  with  Kelley  (1968)  and  Poulton  (1974) .  However 
instead  of  a  linear  ordering  of  control  loop  hierarchy  as  proposed  by 
Kelley  (1968),  Rasmussen's  methodology  is  distinguished  by  three  levels 
of  operator  behavior:  skill-based  behavior,  rule-based  behavior,  and 
knowledge-based  behavior.  These  three  levels  focus  on  the  behavior  of 
the  human  and  not  whether  the  human  is  tracking  a  zero,  first,  or  second 
order  control  system.  Rasmussen's  model  assumes  that  the  order  of  the 
control  system  is  properly  suited  to  the  capabilities  of  the  human 
operator  and  that  decisions  can  be  made  at  any  of  the  three  behavior 
levels.  Skilled-based  behavior  is  defined  as  highly  practiced,  inner- 
loop,  automatic,  nearly  effortless  behavior  and  is  characteristic  of 
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manual  control.  Rule-based  behavior  is  defined  as  a  series  of  rules 
taught  to  an  operator  (i.e. instrument  student)  that  may  eventually 
transition  to  skill-based  behavior.  Rule-based  behavior  is  the  closest 
approximation  to  the  outer- loop  decision  processing  strategy  proposed  by 
Kelley  (1968),  and  Poulton  (1974). 

The  human  operator  can  be  considered  an  adaptive  controller 
(Kelley,  1968);  control  is  maintained  in  the  face  of  change.  Adaptive, 
as  used  by  Kelley  (1968) ,  is  synonymous  with  the  classical  control 
theory  term  of  optimal  control  (Kleinman,  Baron,  &  Levison,  1970) . 
Optimal  control  can  only  be  achieved  if  there  is  a  comparison  between 
the  present  state  and  optimal  state.  The  optimal  control  model  is  a 
human  operator  model  that  provides  a  differential  weighting  schema  for 
the  variables  manipulated  by  the  model.  The  optimal  in  optimal  control 
refers  to  the  strategy  that  a  human  operator  would  use  in  minimizing 
control  errors  by  adjusting  control  gains  and  weightings  depending  upon 
the  situation. 

Optimal  control  modeling  has  been  successfully  applied  to  the 
modeling  of  aircraft  flight  directors  in  approach  to  landing  tasks 
(Kleinman  al . ,  1970) .  The  B-1B  Vertical  Situation  Display  flight 
director  steering  cross  is  considered  a  command  or  "quickened"  display. 
This  display  does  not  directly  tell  the  pilot  what  is  happening  to  the 
system  under  control,  but  simply  what  to  do.  The  inner-loop  functions 
have  been  computed  for  the  pilot  freeing  additional  attentional  capacity 
for  flying  the  flight  director  or  attending  to  other  cockpit  tasks. 

This  is  contrasted  with  raw  glideslope  and  localizer  data  (outer-loop) 
which  depicts  the  actual  location  of  the  aircraft  in  relation  to  the 
Instrument  Landing  System.  The  pilot  must  mentally  compare  the  flight 
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director  st  ing  inputs  (command  information)  against  the  raw 
glideslope  and  localizer  data  (status  information)  as  a  system  check  of 
flight  director  performance.  Disagreement  between  the  flight  director 
and  the  raw  glideslope  and  localizer  steering  cues  is  sufficient  reason 
to  abort  the  approach  to  landing. 

Crosscheck  of  raw  data  compared  with  flight  director  steering 
inputs  is  an  instrument  flying  technique  that  all  Air  Force  pilots  are 
instructed  to  perform  during  approach-to-landing.  The  continuous  input 
of  status  information  (outer-loop)  allows  the  pilot  to  integrate 
information  and  decide  on  a  specific  course  of  action.  Status 
information  (outer-loop)  is  provided  to  the  pilot  at  a  hierarchical 
level  that  is  compatible  with  the  level  at  which  the  decision  making 
process  occurs.  The  decision  to  continue  or  abort  an  approach  to 
landing  is  not  made  based  on  inner-loop  information  but  on  outer-loop 
information.  A  human  model  of  operator  performance  presented  by 
Banbridge  (1981)  and  Kelley  (1968)  suggest  that  the  operator  would  use  a 
different  mental  model  as  he  moved  from  inner  to  outer-loop  control . 
Optimal  control  theory  modeling  (Kleinman  al . ,  1970)  supports  this 
theory  of  mental  model  changes  with  changes  in  hierarchy  by  manipulating 
the  control  strategy  and  weighting  of  information  being  input  to  the 
model.  The  pilot's  perception  of  how  well  the  display  is  functioning 
effects  the  confidence  that  the  pilot  has  in  the  display.  This  effects 
the  outcome  of  an  approach  to  landing  in  cases  that  weather  is  at 
minimum  visibility  and  ceiling.  Subjective  dependent  measures  such  as  a 
post-trial  questionnaires  and  the  Subjective  Workload  Dominance 
Technique  (SWORD)  should  compliment  the  performance  dependent  measures 
collected  in  this  study. 
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The  present  study  shall  try  to  quantify  the  impact  of  status 
information  gain  on  pilot  performance  during  an  approach-to-landing 
flying  task.  Part  of  the  problem  with  flying  an  approach-to-landing 
task  is  that  the  pilot  has  no  display  or  true  indication  of  his  actual 
flight  path.  Deviation  from  desired  flight  path  can  only  be  observed 
indirectly  by  noting  the  difference  between  glideslope  or  localizer 
position  error  or  flight  director  steering  inputs.  Better  root  mean 
square  tracking  performance  with  command  plus  status  information  would 
indicate  that  the  pilot  is  benefiting  from  this  additional  status 
information.  Presentation  of  status  information  (outer-loop)  at  a 
hierarchical  level  compatible  with  pilot  decision  making  (outer- loop) 
should  reduce  the  root  mean  square  error  if  the  pilot  is  able  to  benefit 
from  this  information.  The  pilot's  goal  in  flying  an  Instrument  Landing 
System  approach  is  to  minimize  error  on  both  glideslope  and  localizer 
axes  thus  assuring  that  the  aircraft  will  be  properly  positioned  for  the 
landing. 

POSITIONAL  INFORMATION 

Aircraft  positional  information  relative  to  other  aircraft  and  the 
ground  while  conducting  a  navigation  task,  is  considered  a  requirement 
by  pilots.  Pilots  are  willing  to  accept  higher  workload  during  a  flying 
task  if  they  receive  positional  information  beneficial  to  them  (Hughes, 
Hassoun,  Ward,  &  Rueb,  1990) .  The  pilot  conducting  an  Instrument 
Landing  System  approach  is  always  concerned  with  the  aircraft's  position 
during  the  approach.  Critical  locations  such  as  the  Glideslope 
Intercept  Point  and  Decision  Height  are  confirmed  by  the  pilot  at  a  much 
higher  frequency  than  other  sectors  of  the  approach,  due  to  the 
consequences  of  being  off  course.  For  example,  the  pilot  approaching 
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the  Glideslope  Intercept  Point  would  be  alternating  frair.es  of  reference 
between  outside-in  (to  visualize  the  distance  and  direction  from 
Glideslope  Intercept  Point)  and  inside-out  (to  maintain  correct  heading, 
altitude  .and  airspeed) .  The  goal  at  this  point  is  one  of  knowing  how 
many  nautical  miles  from  the  Glideslope  Intercept  Point  the  aircraft  is, 
what  the  assigned  altitude  is  and  what  altitude  the  Glideslope  Intercept 
Point  should  be,  and  what  compensation  is  needed  to  counteract  the 
effects  of  crosswind.  Any  deviation  from  these  parameters  requires  a 
decision  to  be  made  by  the  pilot  as  to  what  system  changes  are  required 
to  reduce  the  deviations  to  acceptable  levels.  While  on  the  glideslope, 
the  pilot  continues  to  change  frames  of  reference  typically  viewing 
flight  director  data  inside-out  while  viewing  other  information  such  as 
the  distance  in  nautical  miles  the  aircraft  is  from  Decision  Height  or 
touchdown  as  outside-in.  Frame  of  reference  changes  occurs  continuously 
during  an  approach  to  landing  in  instrument  conditions.  Unlike  visual 
flight  conditions  where  the  pilot  is  easily  able  to  determine  his 
position  by  looking  out  the  window,  during  instrument  flight  the  pilot 
must  rely  on  the  instruments  inside  the  cockpit  for  positional 
information.  Crosswind  has  its  greatest  effect  during  the  approach  to 
landing  phase  of  flight,  wind  direction  and  velocity  change  constantly 
as  altitude  and  aircraft  heading  change. 

One  unfortunate  aspect  of  most  flight  directors  is  that  they  were 
designed  as  an  inside-out  instrument.  However  conceptualization  of  the 
aircraft's  position  requires  outside-in  perspective  of  the  world. 
Information  such  as:  where  am  I,  where  should  I  be,  and  where  am  I 
going,  can  not  be  obtained  from  the  flight  director.  The  Attitude 
Direction  Indicator  is  typically  represented  as  inside-out  display,  the 
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pilot  is  sitting  in  the  aircraft  looking  out.  The  aircraft  icon  on  the 
Attitude  Direction  Indicator  is  fixed  and  the  horizon  moves  around  the 
icon.  An  outside-in  display  provides  a  perspective  of  being  external  to 
the  aircraft  looking  in.  xne  aircraft  flight  director  steering  cross 
does  not  provide  that  external  perspective,  and  conversely  provides  the 
perspective  of  inside-out,  moment-to-moment  information  to  the  pilot. 

The  Horizontal  Situation  Indicator,  marker  beacons,  Distance  Measuring 
Equipment,  and  compass  locators  all  attempt  to  provide  the  pilot  with 
the  positional  information  that  is  lacking  in  a  more  centralized  display 
form.  The  Horizontal  Situation  Indicator  provides  the  lateral  cue  as  to 
left/right  while  the  other  cues  inform  the  pilot  of  where  horizontally 
the  aircraft  is  located,  the  altimeter  provides  the  vertical  or  height 
cue.  Subjective  pilot  comments  and  performance  data  observed  in 
simulators  (Wickens  1984)  and  aircraft  suggest  that  pilots  switch  states 
from  outside-in  to  inside-out  perspective  and  then  back  again.  This  may 
account  for  the  reported  occurrence  of  pilots  who  have  been  observed 
misinterpreting  the  Attitude  Direction  Indicator  and  inputting  an 
incorrect  control  input  (Roscoe;  1968,  1980,  Wickens,  sL  id.,  19b.., . 

Pilot  control  reversals  have  been  attributed  to  lack  of  a  stable 
frame  of  reference  (Wickens,  1984)  available  to  pilots.  If  the  pilot  is 
visualizing  information  from  an  outside-in  perspective  (glideslope  and 
localizer  dots)  and  acting  on  information  presented  in  an  inside-out 
perspective  (Attitude  Direction  Indicator) ,  incorrect  pilot  control 
inputs  could  result.  Aircraft  positional  information  in  relation  to 
other  aircraft  as  well  as  navigation  aides  (Smith,  Ellis,  &  Lee,  1984) 
has  been  reported  as  critical  to  pilots.  There  appears  to  be  limited 
information  that  addresses  the  issue  of  positional  information  in  an 
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aircraft  navigational  context.  The  use  of  dots  in  this  study  to 
represent  glideslope  and  localizer  positional  information  may  not  be  the 
most  optimal  method  of  providing  the  outside-in  perspective.  However, 
this  method  is  the  typical  representation  of  glideslope  and  localizer 
information  that  is  provided  to  almost  all  civilian  and  military 
aircraft  flying  today.  The  dot  representation  of  glideslope  and 
localizer  data  help  to  bound  the  area  that  the  aircraft  should  be  within 
during  an  instrument  approach.  The  location  of  the  dot  cursor,  the  rate 
of  cursor  movement  and  the  direction  of  travel  are  available  sources  of 
information  to  the  pilot  during  an  instrument  approach.  Outside-in 
information  provided  by  the  dots  is  only  related  to  the  beamwidths  of 
the  localizer  and  glideslope. 

The  purpose  of  this  research  effort  was  to  evaluate  how  pilots  use 
the  information  content  of  two  modified  navigation  formats  displayed  on 
the  B-1B  Vertical  Situation  Display  during  simulated  approach-to- 
landings  with  and  without  crosswind  conditions.  The  effects  of  command 
and  status  plus  command  information  were  provided  to  the  pilots  for 
evaluation . 

One  hypothesis  is  proposed  for  this  study.  The  hypothesis  posits 
that  the  addition  of  glideslope  and  localizer  raw  data  (status 
information)  to  the  flight  director  (command  display)  will  be 
confirmatory  and  reinforce  the  flying  strategy  of  the  pilot.  Pilot 
performance  using  raw  data  and  flight  director  information  will  be 
better  (less  root  mean  square  error)  than  with  only  the  flight  director. 
Outer-loop  information  provided  by  the  glideslope  and  localizer  dots 
will  provide  trend  and  position  information  that  is  not  available  from 
the  flight  director. 
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SECTION  2 


METHODS 

SUBJECTS 

Twelve  proficient  and  alert-qualified  United  Stated  Air  Force  B-13 
pilots  served  as  subjects  for  this  study.  The  subjects  were  male  and 
had  a  mean  of  1153  flying  hours  (range=  367  to  4800  hrs) in  a  variety  of 
different  aircraft  and  were  instrument  certified  in  the  B-1B  aircraft. 
All  subjects  regularly  flew  the  Engineering  Research  Simulator  for  pilot 
proficiency  as  well  as  the  B-1B  Weapon  Systems  Trainer  and  the  aircraft. 
APPARATUS 

A  fixed-base  B-1B  Engineering  Research  Simulator  located  at  Grand 
Forks  Air  Force  Base,  North  Dakota,  was  used  to  evaluate  pilot 
performance  with  two  Instrument  Landing  System  displays.  The  simulator 
had  neither  an  out -the- window  visual  system  nor  motion  base,  but  it 
fully  modeled  the  displays  and  controls,  flight  and  instrument 
characteristics  of  the  B-1B  bomber  (Purvis,  Reynolds,  &  Marshak,  1990) . 
Each  pilot  flew  a  total  of  16  (8  Dot  and  8  No  dot)  simulated  Instrument 
Landing  System  approaches  to  Grand  Forks  AFB  ILS  runway  35.  The  initial 
conditions  of  the  simulator  at  the  beginning  of  each  trial  were:  (a) The 
aircraft  was  configured  for  an  approach  to  landing,  (b)  the  proper 
approach  airspeed  and  angle  of  attack  were  set  for  the  approach,  (c)  the 
landing  gear,  flaps  and  slats  were  down  and  locked,  and  (d)  engine  power 
was  appropriately  adjusted  for  the  approach. 
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EXPERIMENTAL  DESIGN 


A  2  X  2  X  2  repeated-measures  factorial  design  was  used.  The 
three  factors  were:  Display  type  (dots  showing  glideslope  and  localizer 
raw  data  vs.  no  dots),  wind  condition  (winds  on  or  o.f),  and  approach 
initialization  point  (8  vs.  18NM) .  The  raw  data  display  had  five  fixed 
dots  along  the  left  side  (vertically)  of  the  Vertical  Situation  Display 
representing  glideslope  beam  orientation.  Five  fixed  dots  were  centered 
(horizontally)  along  the  bottom  of  the  Vertical  Situation  Display, 
representing  localizer  beam  orientation.  A  moving  symbol  (triangle), 
adjacent  to  the  dots,  represented  the  centerline  of  the  steering  beam. 
The  wind  was  either  active  at  a  45  degree  angle  of  intercept  to  the 
localizer  at  a  velocity  of  24  knots  diminishing  to  10  knots  at  decision 
height  or  it  was  not  active.  A  complete  wind  model  is  provided  in 
Appendix  B.  Wind  was  provided  because  it  significantly  determined  the 
difficulty  level  of  the  approach  and  increased  realism.  The  initial 
approach  condition,  as  seen  in  Figure  11,  was  either  8  nautical  miles 
from  touchdown  at  an  angle  of  20  degrees  to  the  localizer  beam  or  18 
nautical  miles  at  a  55  degree  angle  to  the  localizer  beam.  The  shorter 
approacn  was  considered  the  more  difficult  of  the  two  approach 
conditions  because  the  aircraft  would  intercept  the  glideslope  and 
localizer  almost  immediately  upon  simulator  release. 
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Not  to  3c*l« 

Figure  11.  Initial  Starting  Conditions 
Localizer  intercept  angle  and  distance  also  directly  affect 
approach  difficulty.  Steep  angles  of  localizer  intercept  make  for  a 
more  difficult  approach  due  to  the  likelihood  of  overshooting  the 
localizer.  Short  approach  distances  before  glideslope  intercept  also 
increase  the  approach  difficulty  due  to  the  limited  amount  of  time 
available  before  capturing  the  glideslope. 

Six  simulator  dependent  variables  were  collected  in  this  study. 
These  variables  were:  1)  glideslope  deviation,  2)  localizer  deviation, 
3)  lateral  alignment  at  decision  height,  4)  deviation  from  target 
airspeed  (in  knots) ,  5)  Roll  rate  variability  and  6)  Pitch  rate 
variability.  An  expanded  explanation  of  each  dependent  variable  is 
provided  in  Appendix  A.  Prior  research  (Roscoe,  1968;  Inderbitzen, 
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Miller,  &  Wiener,  1989)  has  found  that  these  dependent  variables  we re 
the  most  sensitive  of  the  available  flight  parameters.  Five  subjective 
dependent  measures  were  collected  in  this  study.  These  measures  were: 

1)  a  confidence  index  (collected  before  each  subject  flew  the 
simulator),  2)  SWORD,  3  &  4)  a  short  questionnaire  administered 
following  each  block  of  trials,  and  5)  a  nine  question  comparative 
questionnaire  administered  following  test  completion.  SWORD  is  a 
subjective  workload  technique  that  requests  a  subject's  estimation  of 
workload  experienced  utilizing  one  system  as  compared  to  another  system, 
employing  the  same  or  similar  tasks.  SWORD  is  a  paired-comparison 
technique  (  Vidulich,  Ward  &  Schueren  1991)  and  the  pilots  compare  each 
of  the  pairs  formed  by  the  8  experimental  conditions  used.  The  response 
scale  is  composed  of  two  scales  with  nine  levels  of  dominance  that  can 
favor  either  task.  SWORD  data  is  next  converted  into  a  judgment  matrix 
which  is  normalized  and  as  such  has  a  range  of  between  zero  and  one. 

The  calculation  of  SWORD  ratings  is  obtained  from  this  matrix. 

The  Instrument  Landing  System  as  seen  in  Figure  12  becomes 
progressively  narrower  as  you  travel  closer  to  the  beam  origin.  In  this 
study,  the  degree  of  displacement  from  the  centerline  of  the  glideslope 
and  localizer  beams  during  the  approach  was  measured.  Greater 
variability  was  expected  during  the  initial  capture  of  the  localizer. 
These  data  were  collected  for  trend  information  but  no  analyses  were 
performed  until  the  aircraft  was  established  on  the  localizer  beam.  As 
the  aircraft  approaches  the  origin  of  the  localizer  beam,  beamwidth  (in 
feet)  is  decreased  due  to  its  proximity.  A  tighter  tolerance  was 
expected  to  be  applied  here  due  to  the  consequences  of  being  off  axis  on 
either  glideslope  or  localizer.  Air  Force  Manual  (AFM)  51-37  provides 


guidance  as  to  what  tolerance  is  acceptable  before  the  pilot  is  required 
to  execute  a  missed  approach.  Specified  tolerance  is  one  dot  low  or  two 
dots  high  on  glides lope  or  full  scale  deflection  of  the  localizer 
display  needle. 
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PROCEDURE 


Briefing  Pha.se  Each  pilot  received  a  briefing  prior  to  flying  the 
simulator  for  display  evaluation.  The  briefing  consisted  of  a 
description  of  the  changes  made  to  the  existing  B-3D-  Instrument  Landing 
System  display  during  which  he  also  was  given  a  handout  illustrating  the 
changes.  The  purpose  of  the  study  was  explained  and  he  was  told  the 
approximate  amount  of  time  required  for  his  participation.  Next  a 
facility  orientation  and  safety  briefing  was  conducted  followed  by  the 
signing  of  a  standard  Armstrong  Laboratories  consent  form.  A  copy  of 
the  Armstrong  Laboratory  consent  form  is  provided  in  Appendix  C.  After 
signing  the  consent  form  the  pilot  filled  out  a  short  confidence 
questionnaire . 

The  pilots  were  instructed  to  perform  the  simulated  Instrument 
Landing  System  approaches  as  precisely  as  possible.  Emphasis  was  placed 
on  attention  to  beam  alignment,  roll/pitch  rates  and  maintenance  of 
target  airspeed. 

Testing  Phase  The  pilot  was  free  to  ask  any  questions  prior  to 
the  start  of  testing.  The  Horizontal  Situation  Indicator  was  covered  so 
that  only  Distance  Measuring  Equipment  information  was  exposed.  This 
was  done  to  eliminate  any  source  of  raw  dot  information  except  for  that 
provided  on  the  Vertical  Situation  Display. 

Normally  all  pilots  are  required  to  fly  the  simulator  at  least 
once  per  month  to  maintain  their  instrument  proficiency.  Nonetheless, 
all  pilots  had  to  fly  two  practice  Instrument  Landing  System  trials 
before  the  beginning  of  each  block  of  8  trials.  A  short  break  was 
provided  then  testing  resumed  with  two  practice  trials  followed  by  8 
test  trials.  The  two  conditions  tested,  display  with  raw  data  in  dot 
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form  or  no  raw  data,  were  separated  into  two  blocks  of  8  trials.  Once  a 
subject  started  a  block,  all  trials  were  either  dot  or  no  dot 
conditions.  Blocks  were  counterbalanced  across  subjects.  Twelve 
random  patterns  of  16  treatment  conditions  (2  replications  of  the  8 
experimental  cells)  were  implemented  for  this  study.  The  12  testing 
orders  were  randomly  assigned  to  subjects.  Orders  were  determined  so 
that  each  wind  and  distance  condition  occur  equally  often.  The  four 
possible  conditions  occurred  equally  often  in  each  set  of  four  trials. 
Wind  was  presented  in  ABBA  or  BAAB  order  while  distance  was  presented  in 
AABB  or  BABA  order.  The  testing  order  for  each  pilot  is  given  in 
Appendix  D. 

The  experimenters  informed  the  pilot  prior  to  the  beginning  of 
each  trial  what  the  initial  conditions  were,  but  not  whether  wind  was  or 
was  not  present.  After  the  pilot  completed  two  practice  trials,  the 
data  collection  trials  began.  Each  simulator  trial  was  expected  to 
last  between  three  to  six  minutes,  depending  upon  the  initial  starting 
point.  The  simulator  required  approximately  one  minute  to  be  reset.  A 
new  trial  was  then  initiated  upon  the  pilot's  release  command.  After 
completing  all  trials  in  one  block,  the  pilot  was  given  a  short  break 
during  which  he  was  asked  to  complete  a  short  questionnaire  that 
requested  his  opinion  on  the  display  he  had  just  flown.  Once  the 
questionnaire  was  completed,  the  pilot  re-entered  the  simulator  and  the 
retesting  began  on  the  second  block  of  eight  trials.  The  pilot  was 
again  provided  with  two  practice  trials  before  the  actual  testing 
started.  Upon  completion  of  the  second  block  of  trials,  the  pilot  was 
requested  to  complete  the  same  questionnaire  that  was  administered  after 
the  first  block  of  trials.  Once  this  short  questionnaire  was  completed, 
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and  decision  height  was  reached  at  approximately  0.5  nautical  mile  from 
the  runway  threshold.  Six  dependent  variables  were  analyzed,  but  only 
four  variables  were  statistically  significant.  The  four  dependent 
measures  were:  localizer  deviation,  glideslope  deviation,  pitch  rate 
variability,  and  roll  rate  variability.  Root  mean  square  error  was 
analyzed  by  measuring  the  deviation  (absolute)  from  the  mean  or 
referenced  position.  Airspeed  and  altitude  were  not  sensitive  dependent 
measures  and  are  not  reported. 

Localizer  Deviation  The  addition  of  dots  on  the  display  showing 
raw  status  information  increased  the  mean  root  mean  square  error  made  on 
localizer  performance,  F (1, 10)  =5.45,  p  <.05.  Without  dots  on  the 
display,  the  root  mean  square  localizer  error  was  0.020  degrees,  but 
this  increased  to  0.028  degrees  when  the  dots  were  added.  Thus,  the 
redesigned  display  decreased  performance.  As  Table  1  shows,  the  nearer- 
starting  distance  (8NM)  also  produced  larger  localizer  error  than  the 
farther  starting  distance  did,  F(l,10)  =  24.35,  p  <.001.  Mean  root  mean 
square  localizer  error  was  0.031  degrees  for  the  nearer  condition 
compared  to  0.017  degrees  for  the  farther  condition.  Greater  localizer 
error  was  an  expected  result  for  the  shorter  approach  starting  point  and 
confirmed  the  original  intent  of  increasing  task  difficulty.  Wind 
condition  was  also  important  for  localizer  performance.  As  Table  1 
shows,  mean  localizer  root  mean  sqjare  error  increased  from  0.017 
degrees  without  wind  to  0.031  degrees  when  wind  was  present,  F(1,10)  = 
26.99,  p  <.001. 
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TABLE  1.  Localizer  Root  Mean  square  Error 


LOCALIZER  RMSE  (degrees) 

WTND  ON _ WTND  OFF 


8  NM 

18  NM 

8  NM 

18  NM 

MFAN 

DOTS  ON 

.055 

.019 

.021 

.018 

.028 

DOTS  OFF 

.031 

.017 

.016 

.014 

.020 

As  Figure  13  shows,  performance  with  the  wind  on  and  near  starting 
distance  was  poorer  (0.043  degrees)  than  it  was  for  the  18  nautical  mile 
experimental  condition  (0.018  degrees)  for  either  wind  condition.  The 
wind  X  approach  interaction  was  statistically  significant,  F(l,10)  = 
25.9,  p  <.001.  With  no  crosswind,  performance  was  similar  for  18 
nautical  miles  (0.018  degrees)  and  fcr  8  nautical  miles  (0.016  degrees) . 


Wind  On  Wind  Off 


Approach  Distance 


FIGURE  13.  Localizer  Deviation  for  All  Conditions 
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The  dot  X  approach  and  dot  X  wind  interactions  were  not 
statistically  significant,  F(l,10)  =  4.58,  p  >.05  and  F(l,10)  <  1.0,  p 
>.05,  respectively.  Figure  13  shows  the  interaction  of  dots  X  approach 
X  wind  for  the  dependent  variable  localizer.  This  interaction  was  not 
found  to  be  statistically  significant,  F(l/10)  =  4.27,  p  >.05.  The 
complete  ANQVA  table  can  be  found  in  Appendix  3. 

Glideslope  Deviation  Unlike  the  localizer  deviation,  mean  root 
mean  square  error  for  the  glideslcpe  deviation  was  higher  for  the  no 
dots  display  compared  with  the  dots  display,  F(l,  10)  =6.08,  p  <  .05.  As 
Table  2  shows,  mean  root  mean  square  glideslope  deviation  was  0.070 
degrees  when  the  display  without  dots  was  used.  Adding  dots  to  the  axes 
decreased  deviation  by  0.010  degrees.  Mean  glideslope  deviation  was 
also  higher  for  the  wind  on  condition  (0.069  degrees)  than  for  the  wind 
off  condition  (0.061  degrees),  F(l,10)  =  9.12,  p  <  .01.  Neither  the 
effect  of  approach,  F (1 , 10)  <1.0,  p  >.05,  nor  the  interaction  of  wind  X 
approach,  F(l,10)  =  1.74,  p  >.05  were  statistically  significant.  The 
interactions  of  dots  X  approach,  F(l,  10)  <1.0,  p  >.05  and  dots  X  wind, 
F(l/10)  <1.0,  p  >.05  were  not  statistically  significant.  The 
interaction  of  dots  X  approach  X  wind  was  not  statistically  significant, 
F (1, 10)  =  1.01,  p  >.05. 

TABLE  2.  Glideslope  Root  Mean  square  Across  All  Conditions 
GLIDE  SLOPE  RMSE  (degrees) 


DOTS  ON  .068  .059  .058  .054  .060 

DOTS  OFF  .079  .071  .059  .071  .070 
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Pitch  Pate  Variability  Pitch  rate  variability  differences  were 
not  significant  for  the  dots  on  conpared  to  dots  off  conditions,  F(l,10) 
<1.0,  p  >.05.  Pitch  rate  variability  was  about  the  same  (0.200  degrees) 
for  the  8NM  starting  point  and  the  18NM  starting  point  (0.207  degrees); 
this  difference  was  not  statistically  significant,  F(l,10)  =2.93,  p 
>.05.  Application  of  a  crosswind  produced  greater  pitch  rate 
variability  (0.210  degrees)  compared  to  a  wind  off  condition  (0.197 
degrees).  This  difference  was  statistically  significant,  F(l,10) 

=12.22,  p  <.01.  The  interaction  of  dots  X  approach,  F(l, 10)  <1.0,  p 
>.05,  dots  X  wind,  F(l,  10)  <1.0,  p  >.05  and  dots  X  approach  X  wind, 

F (1/ 10)  =4.87,  p  >  .05,  were  not  statistically  significant.  The 
interaction  of  winds  X  approach  was  significant,  F(l,  10)  =8.76,  p  <.05. 
With  8NM  approaches,  pitch  rate  variability  was  reduced  when  pilots  flew 
without  wind  (0.188  degrees)  conpared  to  the  wind  on  condition  (0.212 
degrees) .  In  contrast,  when  pilots  started  their  approach  18NM  out, 
there  was  only  a  0.002  degrees  difference  (0.208-0.206)  in  pitch  rate 
variability  with  the  wind  on  and  wind  off  conditions. 

Figure  14  shorn  pitch  rate  means  with  all  independent  variables. 
Although  Figure  14  suggests  that  a  Llu.ee  way  interaction  may  be  present, 
the  interaction  of  dots  X  approach  X  wind  was  close,  p  =  .0518  but  did 
not  attain  the  0.05  level  of  confidence. 
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Wind  On 


Wind  Off 


Approach  Distance 


Figure  14.  Pitch  Rate  Variability  with  All  Conditions 
Roll  Rate  Variability  Roll  rate  variability  was  not  significantly 
affected  by  dots  on  or  dots  off  conditions,  F(l,10)  =2.22,  p  >.05.  The 
interaction  of  dots  X  approach,  F(l,  10)  <1.0,  p  >.05,  dots  X  wind, 

F (1, 10)  <1.0,  o  >.05,  and  dots  X  approach  X  wind,  F(l,  10)  <1.0,  p  >.05, 
were  not  significant  for  roll  rate  variability.  However,  pilots  had 
greater  roll  rate  variability  for  the  shorter  starting  position  of  8 
nautical  miles,  (0.202  degrees)  than  the  18  nautical  miles  starting 
position,  (0.172  degrees),  F(l,10)  =104.21,  p  <  .001.  Table  3  shows 
mean  roll  rate  variability  for  all  experimental  conditions. 

Table  3.  Roll  Rate  Variability  with  All  Conditions 
ROLL  RI-1SE  (degrees) 

WIND  ON _ WIND  OFF 


8hM 

181M 

8NM 

18NM 

MEAN 

DOTS  ON 

.224 

.169 

.196 

.185 

.193 

DOTS  OFF 

.204 

.159 

.183 

.177 

.181 

MEAN 

.214 

.164 

.190 

.181 

O 
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The  interaction  of  wind  X  approach  was  also  significant  for  roll 
rate  variability,  F(l,10)  =18.03,  p  <  .01.  Pilot  performance  flying  the 
shorter,  8  nautical  miles,  starting  condition  with  wind  on  produced  the 
greatest  variability.  The  mean  was  0.214  degrees  compared  to  mean  of 
0.190  for  the  same  condition  with  wind  off,  a  difference  of  0.024 
degrees.  At  an  18  nautical  miles  starting  condition,  wind  did  not 
increase  variability,  or  if  anything  decreased  variability  by  0.017 
degrees.  The  interaction  of  wind  X  approach  was  expected  and  confirm 
the  intent  of  increasing  task  difficulty  for  the  shorter  approach  and 
wind  on  conditions.  Unexpectedly  only  approach  produced  a  significant 
main  effect;  wind  did  not.  No  other  main  effects  or  interactions  were 
statistically  significant.  The  complete  ANOVA  table  can  be  found  in 
Appendix  G. 

SUBJECTIVE  DATA 

Table  4  presents  the  SWORD  mean  ratings.  II  2  data  were  analyzed 
using  a  2  X  2  X  2  repeated-measures  ANOVA  with  display  type  (dots  or  no 
dots),  wind  (on  or  off),  and  approach  (8  or  18  nautical  miles)  as 
factors . 

TABLE  4.  SWORD  Means  Across  All  Conditions 
SWORD  MEAN  RATINGS 

WIND  QN _ OFF 

MEAN 

APPROACH  8NM  18NM  8NM  18NM _ 


DOTScON 

.1936 

.1105 

.1321 

.0345 

.1302 

DOTS  OFF 

.1727 

.0883 

.1417 

.0767 

1199 

MEAN 

.1832 

.0994 

.1369 

.0806 

.1250 

GRAND 

MEAN 

.1413 

.1088 
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Pilots  did  not  find  the  two  display  types  to  be  different,  the 
dots  on  display  mean  was  0.130  while  the  no  dots  display  was  0.010 
smaller,  F(l,  10)  <1.0,  p  >.05.  However,  pilots  reported  that  the  8 
nautical  miles  starting  point  had  produced  higher  workload.  Starting 
point  was  found  to  be  significant,  F(l,  10)  =  23.29,  p  <  .001.  The  mean 
was  0.160  for  8  nautical  miles  compared  to  0.090  for  the  18  nautical 
miles.  Workload  was  also  reported  as  being  higher  for  the  wind  on 
condition.  Wind  was  found  to  be  significant  F(l, 10)  =9.14,  p  <.05. 

The  mean  was  0.141  for  winds  0"  compared  to  0.109  for  the  no  wind 
condition.  Interaction  of  starting  point  X  wind  was  significant, 

F(l/10)  =6.12,  p  <.05.  Pilots  reported  higher  workload,  mean  of  0.183, 
for  the  wind  on  condition  at  the  8  nautical  miles,  uhan  for  the  18 
nautical  miles  wind  off  starting  point,  mean  of  0.081.  Worxload  was 
also  reported  as  being  higher  for  the  wind  off  condition  at  the  8 
nautical  miles  starting  point,  mean  of  0.137,  compared  to  the  wind  on, 

18  nautical  miles  starring  point,  mean  of  0.099.  Figure  15  illustrates 
that  the  highest  pilot  workload  racing  was  provided  for  the  short 
approach  condition  with  wind.  The  next  highest  level  of  pilot  workload 
was  reported  for  the  short  approach  with  no  wind.  The  longer,  18 
nautical  miles  approaches  received  the  lowest  workload  ratings  with  the 
18  nautical  miles  wind  off  condition  receiving  the  lowest  rating.  The 
short  approach  condition  with  wind  on  was  expected  to  produce  higher 
workload  ratings  and  the  lowest  worklo^u  rating  was  expected  to  be  the 
18  nauti;al  miles  condition  with  no  wind  present.  SWORD  rating  agree 
with  the  general  trend  that  higher  workload  is  produced  with  short 
approaches  and  wind  on.  SWORD  ratings  for  longer  approaches  are  in 
general  agreement  with  most  of  the  performance  data.  No  other  main 
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effects  or  interactions  were  statistically  significant.  The  complete 
ANOVA  table  can  be  found  in  Appendix  G. 


FIGURE  15.  SWORD  Ratings,  Approach  X  Wind 

Post -Trial  Questionnaire  At  the  completion  of  each  of  the  two 
block  trials,  a  short  questionnaire  was  administered.  The  following 
paragraphs  briefly  summarize  the  subject's  response  to  the 
questionnaires . 

The  first  three  questions  requested  pilot  likes/dislikes  and 
situational  awareness  importance  during  an  Instrument  Landing  System 
approach.  Six  of  the  twelve  subjects  liked  the  ability  to  concentrate 
on  only  the  flight  director  and  use  other  non  Cathode  Ray  Tube 
instruments  for  the  crosscheck.  However  the  reduction  of  crosscheck 
workload  for  the  no  dot  display  came  at  the  expense  of  lack  of  forceived 
situational  awareness  which  was  considered  important  by  all  pilots. 

Question  4  asked  the  subjects  to  rate  their  workload  based  on  a 
scale  that  ranged  from  low  to  high.  Pilots'  ratings  indicated  stated 
that  their  workload  was  about  the  same  with  both  displays,  dots  mean  of 
7.81  compared  to  the  no  dots  mean  of  7.36.  Question  4  provided  the 
opportunity  to  conduct  a  t-test  on  the  two  rated  display  types  of  dots 
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and  no  dots.  No  difference  between  display  types  was  reported  by 
pilots,  £(10)  =  0.339,  p  >.05. 

Question  5  ana  6  solicited  the  subjects  opinion  if  they  would  fly 
this  display  to  decision  height  and  what  improvements  to  this  display 
could  they  suggest.  Nine  pilots  stated  they  would  fly  this  display  to 
decision  height  without  raw  data  as  long  as  everything  else  appeared 
normal.  One  pilot  said  he  would  not  and  two  others  said  it  was 
conditional  since  it  would  be  uncomfortable.  No  pilot  was  willing  to 
fly  the  no  dot  display  below  decision  height  without  positional 
information.  Improvements  suggested  by  8  pilots  would  be  the 
incorporation  of  the  raw  data  (dots)  into  the  operational  display.  Two 
other  pilots  stated  they  would  like  better  flight  director  damping  and 
one  pilot  requested  a  separate  warning  light  for  decision  height  rather 
than  radar  altimeter  setting  of  200  feet  AGL. 

No  Dot  Post-Trial  Questionnaire  .  An  additional  questionnaire  was 
provided  after  both  displays  had  been  flown  by  the  subjects.  Subjects 
were  requested  to  rate  overall  aspects  of  the  no  dot  display  on  a  scale 
from  (5)-  excellent  to  (1)-  poor. 

Table  5  provides  a  listing  of  questions  one  through  seven  and  an 
averaged  response  to  those  questions  by  the  pilots.  In  general  pilots 
reported  that  the  Instrument  Landing  System  display  was  easy  to  see  and 
understand,  easy  to  track  and  anticipate  localizer  and  glideslope 
steering.  However  situational  awareness  was  rated  only  slightly  better 
than  fair. 
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TABLE  5.  Pilot  Rating  of  the  No  Dots  Display 


Question  Number  Patina 

1  Ease  of  use  from  IP  to  localizer  4.33 

2  Ease  of  use  from  localizer  cap  to  DH  4.75 

3  Anticipation  and  capture  of  glideslope  4.00 

4  Ease  of  use  tracking  the  localizer  3.92 

5  Maintain  situational  awareness  2.75 

6  Size  of  symbology  on  the  display  3.92 

7  Display  update  rate  4.25 


Questions  8  and  9  requested  pilot  workload  based  on  a  low  to  high 
scale.  Question  8  asked  pilots  to  rate  the  workload  from  initial 
starting  point  to  localizer  capture  using  a  scale  of  (very  low) -  (low) - 
(moderate)-  (high) .  Pilots  rating  are  as  follows:  1  rating  of  very  low, 
5  ratings  of  low,  5  ratings  of  moderate,  and  1  rating  of  high.  Question 
9  asked  pilots  to  rate  their  workload  on  the  same  scale  from  localizer 
capture  to  DH.  Pilots  ratings  are  as  follows:  1  rating  of  very  low,  8 
ratings  of  low,  2  ratings  of  moderate,  and  1  rating  of  high  workload. 
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SECTION  4 


DISCUSSION  and  CONCLUSIONS 

The  hypothesis  tested  in  this  study  was  that  a  performance  would 
deteriorate  when  pilots  were  presented  with  Instrument  Landing  System 
data  that  did  not  contain  raw  glideslope  and  localizer  information.  The 
original  expectation  was  that  pilot  performance  would  be  better  for  the 
status  and  command  condition  as  compared  to  the  command  condition  alone. 
This  hypothesis  was  not  supported  and  does  not  agree  with  Garg's  (1988) 
research.  Garg  suggested  that  status  information  should  improve  the 
performance  of  a  command  display  because  the  status  information 
indicated  true  system  error.  The  raw  glideslope  and  localizer  data  was 
presented  on  the  Instrument  Landing  System  display  in  dot  form;  all 
other  presentations  of  Instrument  Landing  System  data  did  not  contain 
dots. 

Significant  main  effects  were  found  for  display  type  dots  compared 
with  no  dots.  The  localizer  data  provided  less  root  mean  square  error 
with  no  dots,  while  the  glideslope  data  provided  less  root  mean  square 
error  with  dots.  Pilots  were  asked  to  provide  their  best  performance 
for  both  of  the  display  types  (dots  and  no  dots)  that  were  presented  to 
them.  Three  possible  performance  outcomes  are  possible:  1  no  difference 
in  pilot  performance  when  dots  were  added,  2  better  performance  as 
pilots  used  the  trend  information  provided  by  the  addition  of  the  dots, 
and  3  worse  performance  if  pilots  followed  the  dots  data  and  disregarded 
the  flight  director  display.  The  results  do  not  support  any  of  these 
strategies  as  being  successful  for  superior  performance. 
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The  magnitude  of  the  error  differences  were  small  for  both 
localizer  and  glideslope.  This  difference  translated  to  less  than  two 
feet  for  both  axes  at  decision  height.  B-1B  aircraft  land  on  runway 
that  are  300  feet  wide:  Therefore,  a  difference  of  only  two  feet  is  not 
considered  important  for  this  application.  Other  applications  that 
require  greater  landing  precision  could  benefit  from  attaining  superior 
performance.  Greater  localizer  error  with  the  dots  on  condition  coupled 
with  decreasing  glideslope  error  data  imply  that  attentional  sharing  may 
be  occurring.  All  pilots  commented  that  they  would  not  fly  an 
Instrument  Landing  System  approach  in  real  instrument  conditions  below 
decision  height,  with  the  flight  director  without  some  indication  of 
situational/positional  information.  The  status  information  (dots 
display)  provides  a  confidence  indication  that  the  flight  director  is 
operating  properly  but  at  a  cost  of  an  increase  in  localizer  deviation. 
This  confidence  could  decide  the  difference  between  completing  an 
approach  to  landing  or  executing  a  missed  approach  when  weather  is  at 
minimum  visibility  and  ceiling  at  decision  height.  Pilots  may  have  been 
comparing  the  trend  information  that  the  dots  were  providing  to  flight 
director  steering  inputs.  The  difference  in  the  error  for  the  localizer 
compared  to  glideslope  is  surprising.  The  beamwidth  for  the  localizer 
is  5.0  degrees  while  the  beamwidth  for  the  glideslope  is  only  1.4 
degrees.  This  is  approximately  a  3:1  ratio  in  sensitivity  for  the  same 
dot  deflection.  Clearly,  if  the  pilots  were  attending  to  the  glideslope 
and  localizer  beams  equally  than  the  error  difference  would  be  expected 
to  be  the  same  proportionality  as  the  sensitivity  of  the  dots.  The 
differences  between  glideslope  and  localizer  root  mean  square  are  small 
for  the  display  independent  variable.  In  fact,  a  review  of  Tables  1  and 
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2  shov;  that  the  localizer  while  three  times  wider  than  the  glideslope 
had  consistently  lower  root  mean  square.  Greater  attention  appears  to 
be  expended  on  the  localizer  axis  than  was  exoended  on  the  glideslope 
axis  as  evidenced  by  the  differences  in  root  mean  square  error  compared 
to  beamwidth  sensitivities. 

For  the  B-1B  flight  director  with  no  dots  display,  greater 
attention  may  have  been  available  to  track  the  flight  director  steering 
inputs.  This  explanation  is  plausible  as  pilots  reported  no  workload 
difference  with  fewer  distractions  when  no  raw  data  was  present. 

However  equivalent  workload  without  the  raw  data  display  does  not  agree 
with  results  reported  by  Garg  (1988)  and  was  not  an  expected  result  of 
this  study.  Garg's  assumption  that  workload  should  be  lower  with  the 
addition  of  status  information  is  based  on  a  survey  done  by  Weirwille 
and  Connor  (1983) .  Weirwille  and  Connor  stated  that  pilot  perception  of 
workload  correlated  directly  with  root  mean  square  activity.  The  root 
mean  square  activity  that  Weirwille  and  Connor  were  referring  to  was  not 
root  mean  square  error  reduction  but  that  of  stick  activity  from  the 
human  operator. 

The  localizer  performance  results  for  the  dots  display  were 
unexpected.  The  original  hypothesis  predicted  that  the  pilot  would  use 
the  raw  data  to  confirm  the  indications  provided  by  the  flight  director 
steering  cross.  Had  the  pilot  relied  exclusively  on  dots  data, 
overshoots  would  have  been  the  observed  as  the  B-1B  aircraft  inertia 
does  not  permit  perfect  performance  when  flying  dots  only.  The  pilot 
models  that  were  suggested  for  interpreting  the  Instrument  Landing 
System  display  do  not  appear  to  adequately  account  for  the  time  delays 
that  are  incurred  for  both  the  mental  information  integration  of  the  raw 
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dots  data  and  the  longer  response  time  resident  in  a  large  aircraft.  A 
time-sharing,  pilot  mental  model,  that  allocates  attention  primarily  to 
the  flight  director  with  limited  attention  allocated  for  trend 
monitoring,  provided  by  the  dots  data,  should  be  implemented  into  an 
Instrument  Landing  System  task  and  evaluated.  This  time-sharing  was  a 
strategy  that  was  reportedly  used  by  most  pilots  in  this  study. 

There  are  major  differences  in  experimental  apparatus  between  the 
study  performed  by  Garg  (1988)  and  the  Instrument  Landing  System  study 
reported  herein.  These  experimental  differences  may  account  for  the 
reported  performance  differences.  The  Garg  (1988)  study  used  a  personal 
computer  based  simulator  with  a  Cathode  Ray  Tube  representing  the 
display  and  the  flight  controls  provided  by  a  simple  two-axis  joy  stick. 
Of  considerable  importance  was  that  in  Garg's  experiment  only 
longitudinal  information  was  provided  as  an  experimental  manipulation 
and  this  axis  was  tracked  with  and  without  the  addition  of  status 
information.  The  three  subjects  used  in  Garg's  experiment  were  pilots 
of  varying  experience  levels.  The  Instrument  Landing  System  study 
however  used  a  high  fidelity  flight  simulator  with  12  highly  experienced 
pilots  trained  to  fly  the  simulator.  The  vehicle  dynamics  of  the  B-1B 
simulator,  control  loading  system  and  display  system  were  faithful 
replications  of  the  real  aircraft.  B-1B  pilots  were  required  to  track 
both  longitudinal  and  lateral  axes  during  the  Instrument  Landing  System 
approach.  The  experimental  differences  between  these  two  studies  are  of 
great  enough  magnitude  to  produce  different  results. 

When  asked  which  display  was  easier  to  use,  pilots  commented  that 
the  flight  director  with  no  dot  data  was  easier.  However  all  pilots 
stated  that  they  would  not  fly  a  flight  director  only  approach  in  a  real 
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B-1B  unless  there  was  positional  data  available  in  some  form.  Pilots 
appear  willing  to  accept  greater  deviation  from  Instrument  Landing 
System  centerline  and  greater  workload  for  the  added  confidence  that 
positional  awareness  provides.  This  result  agrees  with  Hughes  sl 
aL .  (1990)  finding  on  situational  awareness  in  air  to  air  engagements. 

Subject  four  was  removed  from  the  analysis  because  his  performance 
using  the  status  and  command  display  was  over  five  standard  deviations 
from  the  means  of  the  other  subjects.  If  subject  four's  performance 
could  be  extrapolated  to  the  pilot  population  as  a  whole  then  we  would 
expect  eight  and  one-half  percent  of  the  pilot  population  would  have 
difficulty  flying  an  integrated  status  and  command  display.  Subject 
four's  performance  on  the  command  only  display  was  in  the  same  range  as 
that  of  the  other  subjects.  All  subjects  were  provided  with  the  same 
instructions  and  all  subjects  appeared  to  comply  with  these 
instructions.  However  a  conclusion  can  be  reached  that  subject  four 
decided  to  fly  the  status  and  command  display  in  difference  to  the 
instructions  provided.  This  disregard  of  experimenter  instructions  has 
been  observed  by  the  author  during  the  conduct  of  other  piloting 
experiments  Purvis  al .  (1988) .  Based  on  this  information,  subject 
four  has  been  treated  as  an  outlier  and  it  is  believed  that  he  does  not 
and  is  not  representative  of  the  pilot  population  as  a  whole. 

Poor  pilot  acceptance  of  the  flight  director  only  display  is  a 
paradox.  Pilots  seem  willing  to  accept  higher  workload  and  lower  or 
equal  levels  of  performance  as  a  trade-off  for  positional  or  situational 
information  they  feel  is  an  essential  element  of  the  Instrument  Landing 
System  display. 

The  results  of  this  research  project  does  not  support  the 
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hypothesis  that  performance  will  be  improved  and  workload  will  be 
reduced  with  the  addition  of  status  information  to  a  command  display. 

It  is  possible  that  the  independent  variables  and  measures  that  were 
collected  and  analyzed  for  this  study  were  not  sensitive  to  true 
differences  between  the  two  display  types.  The  independent  variables 
were  selected  based  on  previous  research  on  this  topic.  Had  the 
experimental  set-up  focused  on  a  more  lengthy  and  difficult  approach  to 
the  Instrument  Landing  System,  and  one  that  required  much  higher  levels 
of  positional  awareness  by  the  pilot,  the  results  may  have  been 
different . 

Recommendations  based  on  the  results  of  this  study  support 
retention  of  status  information  with  flight  director  command  steering 
information.  Aircraft  conducting  an  instrument  approach  to  landing 
without  the  generous  300  feet  wide  by  12,500  feet  long  runways,  could 
benefit  from  the  greater  precision  offered  by  the  addition  of  the  dots 
display.  This  recommendation  to  retain  dots  is  based  solely  on  the 
subjective  opinion  of  the  pilots.  Performance  and  SWORD  data  do  not 
show  a  clear  trend  of  one  display  superior  to  another.  Status 
information  as  presented  in  dot  form  on  the  Vertical  Situation  Display 
does  not  translate  situational  context  well.  Translation  of  aircraft 
position  requires  the  pilot  to  mentally  transform  information  presented 
to  him  in  plan  and  profile  views  from  his  approach  plate  to  a  dot 
display  orientation.  However  newer  jet  aircraft  avionics  are  breaking 
away  from  the  tradition  of  the  dot  display  and  are  presenting  aircraft 
position  on  an  Instrument  Landing  System  in  the  same  plan  and  profile 
views  as  that  of  the  instrument  approach  plate. 
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APPENDIX  A.  Definition  of  Dependent  Variables 


1.)  Glideslooe  deviation  .  This  is  defined  as  the  aircraft 
deviation  from  the  center  of  the  GS  beam.  This  can  be  measured  in  either 
distance  from  centerline  or  degrees  from  centerline.  This  study  will  be 
measuring  deviation  in  degrees  from  centerline. 


GS  RMS  =l/n>i£(X-T)**2  t  =  target  value 


2.)  Localizer  deviation  .  This  is  defined  as  the  aircraft  deviation 
from  the  center  of  the  LX  beam.  This  can  be  measured  in  either  distance 
or  degrees  from  centerline.  This  study  will  use  degrees  from  centerline. 


LX  RMS  =l/nVZ(X-T)  **2  T  =  target  value 


3.)  Altitude  Above  Ground  Level  (AGL)  is  the  altitude  above  the 
ground  that  the  aircraft  is  in  feet  at  a  sampling  rate  of  16  hertz 
correlated  with  latitude  and  longitude  posit- ion. 

AGL  =  NUMBER  OF  SAMPLES  PER  SECOND  /  16  HERTZ 


4.)  Knots  Indicated  Airspeed  (KIAS)  .  This  is  a  measure  of  airspeed 
deviation  from  target  or  ideal.  The  VSD  provides  an  indicator  of  target 
airspeed  and  indications  of  too  fast  or  slow.  This  is  measured  in  KIAS. 
For  example,  if  the  target  airspeed  was  167  KIAS  and  the  pilot  flew  180 
KIAS  then  the  reported  difference  is  13  KIAS. 


KIAS  =  MEAN  AND  STANDARD  DEVIATION  /  NUMBER  OF  APPROACHES 


5.)  Roll  jate  .  This  is  the  rate  of  rotation  about  the  longitudinal 
axis  of  the  aircraft.  Roll  is  a  first  order  measure  and  as  such  is 
usually  measured  in  degrees  per  second  or  radians  per  second.  This  study 
measured  roll  rate  in  degrees  per  second. 


*  ROLL  RATE  ^MS  =  l/rW£(X-X)  **2 
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6.)  Pitch  rate  .  Pitch  rate  is  the  rate  of  rotation  about  the 
lateral  axis  of  the  aircraft.  This  is  usually  measured  in  degrees  per 
second  as  it  too  is  a  first  order  system.  Pitch  rate  was  measured  in 
degrees  per  second. 


*  PITCH  RATE  RMS  =  1/.W£(X-X)  **2 


*  X1,X2,X3, . . .Xn  =  a  sample  of  discrete  scores 

n  =  number  of  samples 

lX-Xil  =  deviation  (absolute)  from  the  mean 


Airspeed  Definitions 


Ground  soeed  is  defined  as  the  speed  of  the  aircraft  over  the  ground. 

Indicated  airspeed  is  defined  as  the  airspeed  value  obtained  independent 
of  wind  effects.  This  airspeed  is  usually  read  directly  from  the 
airspeed  indicator. 

True  airspeed  is  defined  as  the  airspeed  that  is  obtained  when  effects 
of  temperature,  barometric  pressure  and  altitude  are  accounted  for. 

A  knot  is  defined  as  a  measure  of  distance  traveled  over  time.  One  knot 
equals  1.15  miles  per  hour. 
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APPENDIX  B.  Wind  Model  for  B-1B  Simulation 


REAL* 4  R_WIND_VEL_CURR 

REAL* 4  K_WIND_VEL_INIT,  K_WIND_HDG 

REAL* 4  K_ALT_INIT,  K_ALT_FINAL,  K_WIND_VEL_FINA 

PARAMETER  (K_ALT_INIT  =  3275.) 

PARAMETER  (K_ALT_FINAL  =  200.) 

PARAMETER  (K_WIND_VEL_INIT  =  40.)  COMMENTS:  TOTS  TS  THE  VELOCITY  AT 
■3275-FEET 

PARAMETER  (K_WIND_VEL_FINAL  =  15.)  COMMENTS :  THIS  THE  VELOCITY  AT  THE 
END  OF  THE  RUN  (APPROXIMATELY  200  FEET) 

PARAMETER  (K_WIND_HDG  =  45/57.29578)  :  THERE  ARE  57.29578  DEGREES  PER 

RADIAN  COMMENTS :  THIS  IS  THE  WIND  DIRECTION 

IF  (WINDS_On)  THEN 

R_WIND_VEL_CURR=  K_WIND_VEL_INIT  -  ( (K_ALT_INIT- 
ALTITUDE_AGL) * (K_WIND_INIT  -  K_WIND_VEL_FINAL)  /  (K_ALT_INIT  K_ALT_FINAL) ) 

IOF_NORTH_STEADY__WIND  =  R_WIND__VEL_CURR  *  COS (K_WIND_HDG) 
IOF_EAST_STEADY_WIND  =R_WIND_VEL_CURR  *  SIN (K_WIND_HDG) 

ELSE 

IOF_NORTH_STEADY_WIND  =0. 

IOF_EAST_STEADY_WIND  =0.ENDIF 

RETURN 

ELD 


APPENDIX  C:  Armstrong  Lab  Consent  Form 


B  IB  ILS  VSD  EVALUATION 
Protocol  No.  90-09 

INFORMATION  PROTECTED  BY  THE  PRIVACY  ACT  OF  1974 
CONSENT  FORM  Initials  _ 

Title:  B-1B  Instrument  Landing  System  Evaluation  (718410) 

OVERVIEW 

1.  The  Instrument  Landing  System  Evaluation  is  set  up  to  collect  pilot 
performance  data  during  routine  Instrument  Landing  System  (ILS) 
approaches  using:  a)  the  flight  director  corrputer  (FDC)  steering 
commands  and  b)  FDC  steering  commands  with  raw  glideslope  and 
localizer  information.  The  purpose  of  the  evaluation  is  to  determine 
if  there  is  any  significant  pilot  performance  difference  between 
making  an  ILS  approach  with  FDC  control  laws  with  and  without  the 
indication  of  raw  localizer  and  glideslope  information.  The  duration 
of  the  ILS  evaluation  will  be  one  week  and  each  subject  will  be  asked 
to  participate  for  one  session  lasting  about  four  hours. 

2.  Each  subject  will  be  asked  to  fly  twenty  ILS  approaches  in  the  B-1B 
Engineering  Research  Simulator  (ERS) . 

3.  The  subject  may  experience  fatigue  due  to  the  length  of  the  study, 
approximately  four  hours,  and  the  number  of  ILS  approaches  required 
for  the  study. 

4.  Participation  in  this  study  will  afford  the  subjects  an  opportunity 
to  see  firsthand  the  proposed  changes  to  the  B-1B  ILS  display  and  also 
give  them  an  opportunity  to  make  comments  about  the  design. 

5.  The  subjects  will  be  advised  of  the  two  different  ILS  display 
configurations  so  they  will  be  able  to  complete  the  evaluation 
questionnaire . 


PLEASE  CAREFUTLY  READ  AND  FILL  IN  THE  FOLLOWING  SECTION 

6.  I,  _  ,  am  participating  because  I  want  to. 

The  decision  to  participate  in  this  research  study  is  completely 
voluntary  on  my  part.  No  one  has  coerced  or  intimidated  me  into 

participating  in  this  program. _ _ 

has  adequately  answered  any  and  all  questions  I  have  asked  about  this 
study,  my  participation,  and 
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the  procedures  involved,  which  are  set  forth  above,  which  I  have  read. 

I  understand  that  the  Principal  Investigator  or  his  designee  will  be 
available  to  answer  any  questions  concerning  procedures  throughout 
this  study.  I  understand  that  if  significant  new  findings  develop 
during  the  course  of  this  research  which  relate  to  my  decision  to 
continue  participation,  I  will  be  informed.  I  further  understand  I 
may  withdraw  this  consent  at  any  time  and  discontinue  further 
participation  in  this  study  without  prejudice  to  my  entitlements.  I 
also  understand  that  the  Medical  Consultant  for  this  study  may 
terminate  my  participation  in  this  study  if  he  feels  this  is  to  be  in 
my  best  interest.  I  may  be  required  to  undergo  certain  further 
examinations,  if  in  the  opinion  of  the  Medical  Consultant,  such 
examinations  are  necessary  for  my  health  or  well  being. 

7.  I  have  considered  and  accept  the  unlikely  but  theoretical 
possibility  as  follows: 

a)  If  physical  exams  and  or  monitoring  of  physiological  parameters 
related  to  this  experiment  are  conducted,  it  is  possible  for  an 
unknown  physical  defect  to  come  to  light  which  might  result  in 
physical  disqualification  from  flight  or  other  special  duty. 

b)  If  physical  injury  were  to  occur  it  could  result  in  physical 
disqualification  from  flight  or  other  special  duty. 

8.  I  understand  that  my  entitlement  to  medical  care  or  compensation  in 
the  event  of  injury  are  governed  by  federal  laws  and  regulations,  and 
that  if  I  desire  further  information  I  may  contact  the  Principal 
Investigator. 

I  understand  that  I  will  not  be  paid  for  my  participation  in  this 
experiment. 

I  understand  that  my  participation  in  this  study  may  be  photographed, 
filmed  or  audio/video  taped.  I  consent  to  the  use  of  these  media  for 
traininq  purposes  and  understand  that  any  release  of  records  of  my 
participation  in  this  study  may  only  be  disclosed  according  to  federal 
law,  including  the  Federal  Privacy  Act,  5  U.S.C  552a,  and  its 
implementing  regulations.  This  means  personal  information  will  not  be 
released  to  an  unauthorized  source  without  my  permission. 
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I  FULLY  UNDERSTAND  THAT  I  AM  MAKING  A  DECISION  WHETHER  OR  NOT  TO 
PARTICIPATE.  MY  SIGNATURE  INDICATES  THAT  I  HAVE  DECIDED  TO 

PARTICIPATE,  HAVING  READ  THE  INFORMATION  PROVIDED  ABOVE. 

Volunteer  Signature  SSAN  Date  Time 


Witness  Signature  Date  Time 

INFORMATION  PROTECTED  BY  THE  PRIVACY  ACT  OF  1974 

Authority  10  U.S.C.  8012,  Secretary  of  the  Air  Fo  e;  powers  and 
duties;  delegation  by;  implemented  by  DOI  12-1.  Office  ocator 

Purpose  is  to  request  consent  for  participation  in  pproved  medical 
research  studies.  Disclosure  is  voluntary. 

Routine  Use  Information  May  be  disclosed  for  any  of  the  blanket 
routine  uses  published  by  the  Air  Force  and  reprinted  in  AFP  12-36  and 
in  Federal  Register  52  FR  16431. 
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APPENDIX  D.  Order  of  Testing 


B-l  Display  Evaluation 

Order  of  Testing  Wind  and  Approach  Conditions 
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Entries  report  wind  and  approach  condition.  For  example,  0N8 
refers  to  WIND  ON  and  an  approach  distance  of  8NM. 
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Questionnaire 


B-1B  IDS  VSO  EVALUATION  QUESTIONNAIRE 

THE  PURPOSE  0?  THIS  QUESTIONNAIRE  IS  TO  OBTAIN  TOUR  OPINION  ABOUT  THE  IDS 
DISPLAY  WITHOUT  RAW  GS  AND  LOT  DOTS. 

1.  HCW  EAST  WAS  THE  DISPLAY  TO  USE  FROM  THE  INITIALIZATION  POINT  TO 
ESTABLISHES  ON  THE  LOCALIZER? 

5  -  EXCELLENT  -  IT  WAS  EAST  TO  USE  AND  WAS  CLEARLY  UNDERSTOOD, 

4  -  COCO  -  IT  WAS  FAIRLY  EASY  TO  USE  AID  FAIRLY  EASY  TO  UNDERSTAtO. 

3  -  ADEQUATE  -  FOR  HOST  ALL  TASKS. 

2  -  SOKE  DIFFICULTY  USING  THE  DISPLAY. 

1  -  POOR  -  HAD  MAJOR  PROBLEMS  USING  THE  DISPLAY. 

2.  HOW  EASY  WAS  THE  DISPLAY  TO  USE  FROM  LOCALIZER  CAPTURE  TO  DECISION 
HEIGHT  (OH)? 

5  -  EXCELLENT,  IT  WAS  EASY  TO  USE  AND  WAS  CLEARLY  UNDERSTOOD. 

4  -  GOOD  -  IT  WAS  FAIRLY  EASY  TO  USE  AND  FAIRLY  EASY  TO  UNDERSTAND. 

3  -  ADEQUATE  -  FOR  HOST  ALL  TASKS. 

2  -  SOKE  DIFFICULTY  USING  THE  DISPLAY. 

1  -  POOR  -  HAD  MAJOR  PROBLEMS  USING  THE  DISPLAY. 

3.  HCW  EASY  WAS  THE  DISPLAY  TO  USE  FOR  ANTICIPATING  AND  CAPTURING  THE  GS? 

5  -  EXCELLENT,  IT  WAS  EASY  TO  USE  A®  WAS  CLEARLY  UNDERSTOOD. 

4  -  GOOO  -  IT  WAS  FAIRLY  EASY  TO  USE  AND  FAIRLY  EASY  TO  UNDERSTAND. 

3  -  ADEQUATE  -  FOR  MOST  ALL  TASKS. 

2  -  SOME  DIFFICULTY  USING  THE  DISPLAY. 

1  -  POOR  -  HAD  MAJOR  PROBLEMS  USING  THE  DISPLAY. 

4.  HOW  EASY  OVERALL  WAS  THE  DISPLAY  TO  USE  FOR  ANTICIPATING  AND  TRACKING  THE 
LOCALIZER? 

5  -  EXCELLENT,  IT  WAS  EASY  TO  USE  AND  WAS  CLEARLY  UNDERSTOOD. 

4  -  GOOD  -  IT  WAS  FAIRLY  EASY  TO  USE  AND  FAIRLY  EASY  TO  UNDERSTAND. 

3  -  ADEQUATE  -  FOR  MOST  ALL  TASKS. 

2  -  SOME  DIFFICULTY  USING  THE  DISPLAY. 

1  -  POOR  -  HAD  MAJOR  PROBLEMS  USING  THE  DISPLAY. 

5.  HCW  WELL  WERE  YOU  ABLE  TO  MAINTAIN  SITUATIONAL  AWARENESS? 

5  -  EXCELLENT,  IT  WAS  EASY  TO  USE  AND  WAS  CLEARLY  UNDERSTOOD. 

4  -  GOOO  -  IT  WAS  FAIRLY  EASY  TO  USE  AND  FAIRLY  EASY  TO  UNDERSTAND. 

3  -  ADEQUATE  -  FOR  MOST  ALL  TASKS. 

2  -  SOME  DIFFICULTY  USING  THE  DISPLAY. 

I  -  POOR  -  HAD  MAJOR  PROBLEMS  USING  THE  DISPLAY. 
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6.  BATE  THE  SIZE  CF  “WE  SYMBOLOGY  USED  CH  THE  DISPLAY. 

5  -  EXCELLENT,  IT  VRS  EASY  TO  USE  AND  WAS  CLEARLY  UNDERSTOOD. 

1  -  GOOD  -  IT  WAS  FAIRLY  EASY  TO  USE  AND  FAIRLY  EASY  TO  UNDERSTAND. 
3-  ADEQUATE  -  FOR  HOST  ALL  TASKS. 

2  -  SOME  DIFFICULTY  USING  THE  DISPLAY. 

1  -  POOR  -  HAD  MAJOR  PROBLEMS  USING  THE  DISPLAY. 

7.  IS  THE  UPDATE  RATE  ADEQUATE  FOR  FLYING  AN  ILS? 

5  -  EXCELLENT,  IT  HAS  EASY  TO  USE  At®  HAS  CLEARLY  UNDERSTOOD. 

1  -  GOOD  -  IT  HAS  FAIRLY  EASY  TO  USE  AtO  FAIRLY  EASY  TO  UNDESSTAJO 

3  -  ADEQUATE  -  TOR  HOST  ALL  TASKS. 

2  -  SOKE  DIFFICULTY  USING  THE  DISPLAY. 

1  -  POOR  -  HAD  MAJOR  PROBLSNS  USING  THE  DISPLAY. 

8.  RATS  YOUR  WORKLOAD  FROM  THE  INITIALIZATION  POINT  TO  TOC  CAPTURE 
VERY  TOW  LOW  MODERATE  Hlffl  VERY  HIGH 


9.  RATE  YOUR  WORKLOAD  FROM  LOCALIZER  CAPTURE  TO  DH. 

VERY  TOW  TOW  MODERATE  HIGH  VERY  HIGH 
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APPENDIX  E.  Continued  Comparative  Questionnaire 


Questionnaire  to  be  administered  a+ter  each  trial  block 

1.  Now  that  >'ou  have  1 1  own  this  display,  what  do  like 
and  dislike  about  it  and  why? 

2.  Was  this  display  use-ful  -for  situational  awareness? 

3.  Is  situational  awareness  important  on  an  ILS  ap¬ 
proach? 

4.  Rate  your  workload  using  this  display.  Mark  an  X  that 
is  closest  to  your  workload  during  the  ILS  approach. 

Low - Med  i  urn - H  i  gh 

5.  Would  you  i 1 ✓  this  display  to  OH?  Disregard  ARM  51-37 
guide  nee  tor  this  question. 

d.  What  do  you  suggest  to  improve  this  display  and  why? 
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APPENDIX  F. 


Subject  Four  Analysis 


MEAN  OF  THE  11  REMAINING  SUBJECTS,  AFTER  DROPPING  SUBJECT  4-  -3.6824 

MEAN  OF  SUBJECT  4-  -  0.6554 
S  -  STANDARD  DEVIATION  OF  THE  11  SUBJECTS  (WITHOUT  SUBJECT  4 ) - 0 . S 7 6 9 

WE  WANT  TO  KNOW  HOW  MANY  st.devs.  AWAY  FROM  THE 
REMAINING  11  SUBJECTS'  MEAN  IS  THE  MEAN  OF  SUBJECT  4. 

i.e.|  -3.6842  -(-  0.6554)  |  -  k  *  (0.5769) 

WHAT  IS  k  ? 

k»3. 0288/0. 5769 
*5.25 

HENCE  MEAN  OF  SUBJECT  4  IS  5.25  ST. DEV.  AWAY  FROM  THE  MEAN  OF  THE 
REMAINING  11  SUBJECTS. 


DOTS 


ON 


WIND  APPROACH 

LOG 

GS 

PITCH 

RATE 

ROLL 

RATE 

8  NM 

1.19746 

.16894 

.29597 

.34102 

ON 

18  NM 

.45802 

.23817 

.26224 

.21889 

8  NM 

.36427 

.21717 

,28440 

.25122 

OFF 

18  NM 

.41215 

.23062 

.25996 

.18491 
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APPENDIX  F.  CON’T. 


DOTS 

WIND  APPROACH 

LOC 

GS 

PITCH 

RATE 

ROLL 

RATE 

ON 

8  NN/i 

.04428 

.07155 

.22791 

.39071 

18  NM 

.02389 

.08708 

.23936 

.45286 

OFF 

8  NM 

.02335 

.07792 

.25406 

.36823 

OFF 

18  NM 

.02032 

.07836 

.19903 

.40851 
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APPENDIX  G.  ANALYSIS  OF  VARIANCE 


REPEATED  MEASURES  ANALYSIS  OF  VARIANCE  RESULTS 
ALL  TESTS  HAVE  1, 11  DEGREES  OF  FREEDOM 
WITHOUT  GROUP  AND  WITH  SUBJECT  4 


LOCALIZER  RMSE  (degrees) 


SOURCE 

SUM  OF 
SQUARES 

F  VALUE 

P  VALUE 

DOTS 

6.759486 

4.37 

0.0606 

WINDS 

5.893286 

32.46 

0.0001* 

APPROACH 

5.380230 

28.40 

0.0002* 

DOTS*WINDS 

0.030625 

0.25 

0.6242 

rX5TS*APPROACH 

0.295430 

4.86 

0.0497* 

WINDS : *  APPROACH 

2.816386 

31.86 

0.0001* 

DOTS  * WINDS  * APPROACH 

0.317634 

5.43 

0.0399* 

GLIDE  SLOPE  RMSE  (degrees) 


SOURCE 

SUM  OF 
SQUARES 

F  VALUE 

P  VALUE 

DOTS 

0.047619 

0.12 

0.7334 

WINDS 

0.357517 

8.57 

0.0138* 

APPROACH 

0.005113 

0.05 

0.8239 

DOTS*WINDS 

0.015782 

0.27 

0.6152 

DOTS  *APPROACH 

0.109031 

0.82 

0.3852 

WINDS*APPROACH 

0.120420 

1.20 

0.2970 

DOTS  * WINDS  *APPROACH 

0.061394 

1.03 

0.3328 

ROLL  RATE  RMSE  (degrees) 


SOURCE 

SUM  OF 
SQUARES 

F  VALUE 

P  VALUE 

DOTS 

0.000068 

0.01 

0.9180 

WINDS 

0.001319 

1.93 

0.1926 

APPROACH 

0.019893 

112.38 

0.0001* 

DOTS* WINDS 

0.000244 

0.61 

0.4507 

DOTS  * APPROACH 

0.002199 

1.73 

0.2158 

WINDS *APPROACH 

0.009222 

18.60 

0.0012* 

DOTS*WINDS*APPROACH 

0.000198 

0.87 

0.3709 

*  =  level  of  confidence  0.05 
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APPENDIX  G.  CON'T. 


REPEATED  MEASURES  ANALYSIS  OF  VARIANCE  RESULTS 
ALL  TESTS  HAVE  1, 11  DEGREES  OF  FREEDOM 
WITHOUT  GROUP  AND  WITH  SUBJECT  4 


PITCH  RATE  RMSE  (degrees) 


SOURCE 

SIM  OF 
SQUARES 

F  VALUE 

P  VALUE 

DOTS 

0.000245 

0.18 

0.6783 

WINDS 

0.004000 

13.21 

0.0039* 

APPROACH 

0.000497 

0.92 

0.3576 

DOTS*WINDS 

0.000009 

0.04 

0.8374 

DOTS* APPROACH 

0.000118 

0.32 

0.5855 

WINDS*APPROACH 

0.001953 

4.99 

0.0472* 

DOTS  * WINDS  * APPROACH 

0.002755 

6.54 

0.0266* 

*  =  level  of  confidence  0.05 
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APPENDIX  G.  CCN'T. 

REPEATED  MEASURES  ANALYSIS  OF  VARIANCE  RESULTS 
ALL  TESTS  HAVE  1, 10  DEGREES  OF  FREEDOM 
WITHOUT  GROUP  AND  WITHOUT  SUBJECT  4 


LOCALIZER  RMSE  (degrees) 


SUM  OF 


SOURCE 

SQUARES 

F  VALUE 

DOTS 

2.072777 

5.45 

WINDS 

5.390935 

26.99 

APPROACH 

5.053018 

24.35 

DOTS* WINDS 

0.019151 

0.15 

DOTS  *APPROACH 

0.301817 

4.58 

WINDS *  APPROACH 

2.497021 

25.91 

DOTS*WINDS*APPROACH  0.273144 

GLIDE  SDPE  RMSE  (degrees) 

SUM  OF 

4.27 

SOURCE 

SQUARES 

F  VALUE 

DOTS 

0.548117 

6.08 

WINDS 

0.389549 

9.12 

approach 

0.029666 

0.31 

DOTS* WINDS 

0.006587 

0.10 

DOTS  *APPROACH 

0.122586 

0.84 

WINDS  * APPROACH 

0.176515 

1.74 

DOTS  * WINDS  * APPROACH 

0.065837 

1.01 

*  =  level  of  confidence  0.05 


P  VALUE 

0.0418* 

0.0004* 

0.0006* 

0.7108 

0.0580 

0.0005* 

0.0658 


P  VALUE 

0.0334* 

0.0129* 

0.5893 

0.7531 

0.3799 

0.2161 

0.3392 
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APPENDIX  G.  CON'T. 

REPEATED  MEASURES  ANALYSIS  OF  VARIANCE  RESULTS 
ALL  TESTS  HAVE  1, 10  DEGREES  OF  FREEDOM 
WITHOUT  GROUP  AND  WITHOUT  SUBJECT  4 


ROLL  RATE  RMSE  (degrees) 


SUM  OF 

SOURCE 

SQUARES 

F  VALUE 

P  VALUE 

DOTS 

0.003358 

2.22 

0.1672 

WINDS 

0.000310 

0.78 

0.3990 

APPROACH 

0.019084 

104.21 

0.0001* 

DOTS*WINDS 

0.000105 

0.25 

0.6249 

DOTS  * APPROACH 

0.000323 

0.61 

0.4540 

WINDS*APPROACH 

0.009347 

18.03 

0.0017* 

DOTS  * WINDS  * APPROACH 

0.000041 

0.22 

0.6518 

PITCH  RATE  RMSE  (degrees) 


SOURCE 

SUM  OF 
SQUARES 

F  VALUE 

P  VALUE 

DOTS 

0.000009 

0.01 

0.9278 

WINDS 

0.003978 

12.22 

0.0058* 

APPROACH 

0.001166 

2.93 

0.1175 

DOTS*WINDS 

0.000010 

0.04 

0.8365 

DOTS*APPROACH 

0.000096 

0.23 

0.6394 

WINDS*APPROACH 

0.002730 

8.76 

0.0143* 

DOTS  *WINDS  *APPROACH 

0.002185 

4.87 

0.0518 

*  =  level  of  confidence  0.05 
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APPENDIX  G.  CON'T. 


REPEATED  MEASURES  ANALYSIS  OF  VARIANCE  RESULTS 
ALL  TESTS  RAVE  1,10  DEGREES  OF  FREEDOM 
WITHOUT  GROUP  AND  WITHOUT  SUBJECT  4 


SWORD  RATINGS 


SOURCE 

DOTS 

WINDS 

APPROACH 

DOTS* WINDS 

DOTS* APPROACH 

WINDS * APPROACH 

DOTS*WINDS*APPROACH 


SUM  OF 
SQUARES 

F  VALUE 

0.002332 

0.09 

0.023270 

9.14 

0.108010 

23.29 

0.002784 

1.86 

0.000478 

0.11 

0.004159 

6.12 

0.000356 

0.76 

* 


P  VALUE 

0.7716 

0.C128* 

0.0007* 

0.2028 

0.7484 

0.0329* 

0.4024 


=  level  of  confidence  0.05 
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